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Charter takeovers are traditional public schools restarted as charter 
schools. We develop a grandfathering instrument for takeover atten- 
dance that compares students at schools designated for takeover 
with a matched sample of students attending similar schools not yet 
taken over. Grandfathering estimates from New Orleans show sub- 
stantial gains from takeover enrollment. In Boston, grandfathered 
students see achievement gains at least as large as the gains for 
students assigned charter seats in lotteries. A non-charter Boston 
turnaround intervention that had much in common with the takeover 
Strategy generated gains as large as those seen for takeovers, while 
other more modest turnaround interventions yielded smaller effects. 
(JEL D44, H75, 121, 128) 


No child’s chance in life should be determined by the luck of a lottery. 
— President Barack Obama 


The National Alliance for Public Charter Schools (NACPS) reports a net increase 
of 1,092 charter schools between fall 2011 and fall 2015, with an enrollment gain 
of 43.6 percent. Charter growth has been especially strong in large urban districts 
where many students are poor and most are nonwhite. The schools in these dis- 
tricts are often described as low-performing, with low standardized test scores and 
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high truancy and dropout rates |! Studies using randomized admissions lotteries to 
evaluate urban charter schools have repeatedly and convincingly shown remarkable 
achievement gains for urban charter lottery winners. The external validity of these 
estimates is less clear, however. 

In the 2014~2015 school year, the New Orleans Recovery School District (RSD) 
became America’s first all-charter public school district. This unique transformation 
offers the opportunity to explore the predictive value of lottery-based charter effects. 
RSD emerged from a 2003 effort to improve underperforming public schools in 
New Orleans, home to some of the worst schools in the country. State legislation 
allowed the Louisiana Department of Education (LDE) to take control of, manage, 
and delegate the operation of low-performing schools to outside operators. New 
Orleans public schools that came under state control became part of RSD, while 
other schools remained under the authority of the Orleans Parish School Board 
(OPSB) HL 

Hurricane Katrina decimated New Orleans’ schools in August 2005. In the after- 
math of the storm, RSD took control of 114 low-performing New Orleans schools, 
leaving OPSB with authority over only 17 of the schools it ran before Katrina. At the 
same time, both RSD and OPSB converted increasing numbers of low-performing 
schools to charters. By fall 2008, when combined RSD and OPSB enrollment had 
reached 36,000 (just over half of pre-Katrina OPSB enrollment), the RSD charter 
share hit 49 percent. Since 2008, RSD charter enrollment growth has accelerated 
further: September 2014 saw the closure of the few remaining direct-run traditional 
public schools in RSD (OPSB continues to operate a mix of traditional and charter 
schools). 

A distinctive feature of New Orleans’ charter expansion is the fact that most of 
the RSD charter schools that have opened since 2008 are takeovers. A charter take- 
over occurs when an existing public school, including its facilities and staff, comes 
under charter management. Importantly, takeovers guarantee seats for incumbent 
students, “grandfathering” these students into the new school. By contrast, most 
charter schools in other districts open as startups, that is, as new schools (some- 
times in existing school buildings), with no seats guaranteed by virtue of previous 
enrollment and an active enrollment process that uses a lottery when schools are 
oversubscribed. 

Boston’s experiment with charter takeovers has unfolded with less urgency than 
New Orleans’, but some of the forces behind it are similar. At the end of the 2010-— 
2011 school year, nine schools in the Boston Public School (BPS) district were 
closed as a consequence of their persistently low performance. Two of these schools 
were replaced by charters: UP Academy Boston replaced the former Gavin mid- 
dle school and Boston Green Academy (BGA) replaced the former Odyssey high 
school. These in-district charter schools, known in the state bureaucracy as Type-III 
Horace Mann schools, mark a new approach to charter authorization and school 


‘Charter schools are publicly funded private schools that operate outside the public sector. See the National 
Center for Education Statistics (NCES 2013) for national enrollment statistics by sector and NACPS (2013, 2014a, 
and 2015) for statistics on charter growth and market share. The Center for Research on Education Outcomes 
(CREDO 2013a) compares the demographic characteristics of traditional public and charter school students; 
NACPS (2014b) gives statistics on charter shares by district. 

? Cowen Institute (2011) outlines the history of RSD. 
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autonomy in Massachusetts. The Boston School Committee authorizes in-district 
charter schools and funds them through the BPS general budget. In-district charter 
teachers are also members of the Boston Teachers Union. Outside of pay and 
benefits, however, terms of the relevant collective bargaining agreements are waived 
and these schools are free to operate according to their charters. Boston’s in-district 
charters opened with new school leaders and new teaching staff, employed on an 
essentially at-will basis, while guaranteeing seats to students formerly at Gavin and 
Odyssey (“legacy schools,” in our vernacular). 

This paper evaluates the causal effects of RSD and Boston takeover schools on 
their students’ achievement using an instrumental variables (IV) strategy that exploits 
the grandfathering provisions used initially to fill takeover seats. By offering a tool 
for the evaluation of the rapidly proliferating charter takeover model, grandfathering 
provides the opportunity to answer new questions about urban school reform. The 
growing set of econometric estimates exploiting charter admissions lotteries con- 
sistently show large gains for students at urban charters, but these estimates neces- 
sarily capture causal effects only for charter applicants—a self-selected population 
that may be especially likely to see gains from the charter treatment. By contrast, 
grandfathered enrollment in charter takeovers is passive: an existing population is 
guaranteed seats in the new school. Takeover experiments therefore identify causal 
effects for students who haven’t actively sought a charter seat. 

In addition to contributing to the long-running charter debate, our empirical 
results are of immediate policy interest. The proliferation of charter takeovers 
reflects a federal push to encourage states to “require significant changes in schools 
that are chronically underperforming and aren’t getting better” (Duncan 2010). The 
FY2011 federal budget addressed this challenge with a dramatic increase in funding 
for School Improvement Grants (SIGs). Federal SIGs, which offer up to two mil- 
lion dollars annually per qualifying school, support three restructuring models; the 
takeover charters studied here qualify for federal support under the “restart”? model 
(US Department of Education 2009). Large urban districts besides Boston and New 
Orleans have also begun experimenting with takeovers. Tennessee’s Achievement 
School District and Michigan’s Education Achievement Authority are modeled 
on RSD, each with a large share of charter takeovers. Philadelphia’s Renaissance 
Initiative has likewise turned many low-performing schools over to charter man- 
agement. A British takeover model has also flowered in the form of England’s 
Academies, conversions of state-run schools that remain publicly-funded but oper- 
ate with charter-like autonomy (Eyles and Machin 2015). 

Our results suggest takeover enrollment boosts achievement by as much or more 
than the gains seen for urban charter lottery applicants. In addition to a detailed anal- 
ysis of takeover treatment effects in Boston and New Orleans, we also look briefly at 
an alternative school restructuring model in Boston, known as a “turnaround.” One 
turnaround intervention was charter-like, replacing most staff with young outsiders 


3 Lottery estimates are reported in, e.g., Abdulkadiroglu et al. (2011); Angrist et al. (2012); Angrist, Pathak, 
and Walters (2013); Dobbie and Fryer (2011); Dobbie and Fryer (2013); Hoxby, Murarka, and Kang (2009); and 
Tuttle et al. (2013). Ravitch (2010, pp. 141-144) and Rothstein (2011) challenge the external validity of charter 
treatment effects estimated using lotteries. See also Rothstein’s account of high scores at KIPP: “They select from 
the top of the ability distribution those lower-class children with innate intelligence, well-motivated parents, or their 
own personal drives, and give these children educations they can use to succeed in life” (Rothstein 2004, p. 82). 
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much like those employed at UP and emphasizing data-driven instruction and stu- 
dent discipline and comportment. Two other middle school turnarounds were more 
modest, involving limited reforms and less staff turnover. The first intervention 
appears to have generated gains as large as those seen at Boston’s in-district charter 
middle school (subsidized, in part, by greater SIG funding), while the other turn- 
arounds yielded less impressive effects. 


I. Background 
A. Why Lottery and Grandfathering Estimates Might Differ 


A stylized sample selection model shows why the effects of charter enrollment 
induced by grandfathering might differ from charter gains identified by admissions 
lotteries. Suppose students face a normally-distributed unobserved net cost of char- 
ter application, denoted 7, applying when 77 < A for some constant threshold A. We 
write the gains from charter attendance in potential outcomes notation as Y, — Yo, 
also assumed to be normally distributed. 

Lottery-generated admission offers, indicated by Z;, are randomly assigned con- 
ditional on application and therefore conditionally independent of 7 and potential 
outcomes. For the purposes of this theoretical discussion we ignore ex post non- 
compliance with offers, assuming that any applicant offered a charter seat takes 
it. This implies that lottery-based comparisons of applicants identify the average 
causal effect for lottery applicants, 


E[Y|Z, = 1,n < A]—E[Y|Z, = 0, < A] = E[Y, —Yln < Al. 


With joint normality of outcomes and costs, the average effect of charter enrollment 
on lottery applicants can be written 


E[Y, — Yoln < A] = E[¥ — Yo] — p(N% — Yo, ACA), 


where p(Y; — Yo, 7) is the coefficient from a regression of gains on costs and \(A) 
is a positive Mills ratio term. 

The selection-on-net-costs model suggests p(Y; — Yo, 7) < 0, since 7 equals 
costs minus benefits. This in turn implies that the average causal effect for lottery 
applicants exceeds the population average charter attendance gain, E|Y, — Y|. In 
other words, as in a simple Roy (1951) model, applicants selected on gains see 
larger causal effects than would be seen in a random sample. 

In the grandfathering scenario, school districts select takeovers from a set of 
candidate schools judged to be underperforming. Suppose that takeover candidates 
have Yo < L, where Y? is a standardized baseline score, assumed here to be con- 
stant within schools, and L is a performance cutoff (e.g., a “Level 4” designation 


+ Oreopoulos (2006) similarly compares the causal returns to schooling parameters identified by alternative 
compulsory schooling instruments. 
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in Massachusetts). Suppose also that takeover events, indicated by Z7, are as good 
as randomly assigned among the set of low-performing candidates (an assumption 
supported by the covariate balance tests discussed below). Conditional on candidacy, 
comparisons by grandfathering eligibility, that is, by Z, identify the average causal 
effect for students with low baseline scores, 


BZ 24,2 29) - 2a 0 Sa) aly ly = 1. 
Again using normality, we have 
E|y,-¥%|¥° < 2] = £[)-%] -— oh —%, PAD), 


where olY, — ¥, ¥") is the coefficient from a regression of gains on baseline scores 
and A(L) is a Mills ratio term. Here too, we ignore ex post noncompliance so as to 
focus on the takeover decision. 

In this model, the correlation between baseline scores and the gains from charter 
enrollment determines the average causal effect identified by grandfathering. We’ve 
seen elsewhere that applicants with low baseline scores often seem to reap espe- 
cially high gains from charter enrollment (e.g., Angrist et al. 2012). Most impor- 
tantly, however, conditional on the baseline score used to gauge low performance, 
the grandfathering instrument identifies a population average treatment effect. 

This discussion shows why the grandfathering identification strategy might gen- 
erate a more representative average causal effect than lottery-based identification 
strategies, at least for populations with similar baseline scores. In practice, however, 
lottery applicants need not be selected on gains (indeed, Walters 2014 finds evidence 
for a kind of “reverse Roy” selection pattern). Ultimately, the relative magnitude of 
lottery- and grandfathering-based estimates is an empirical question, resolved in 
part by the analysis that follows. 


B. Takeovers in New Orleans RSD 


The 2008 school year marked the beginning of a period of relative stability in 
RSD enrollment, leadership, and finances, along with district-wide improvements 
in test scores. RSD achievement gains in both direct-run and charter schools are 
described oe which compares post-2008 math achievement trends in RSD 
and OPSB with all schools in Louisiana. Average achievement for traditional and 
RSD charter students runs mostly below the statewide and OPSB averages, but the 
RSD shortfall was much reduced by 2014. 

Among the RSD charter schools opened since fall 2008 and operating in spring 
2014 (excluding alternative schools that serve special populations), 21 are takeovers 
and 13 are startups. Even by the standards of the heated debate over school reform, 
the proliferation of charter takeovers in New Orleans has proven to be especially 
controversial (see, for example, Darling-Hammond 2012). 


>See Figure B1 in the online Appendix. 
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T T T T 
2008 2010 2012 2014 
Year 


FiGureE 1. MATH ScorES IN RSD AND ELSEWHERE 


Notes: This figure plots the average percentage of RSD, OPSB, and Louisiana students that 
achieve basic or above status on LEAP/iLEAP math exams in fifth-eighth grades. Scores for 
OPSB and Louisiana students are from the Louisiana Department of Education, https://www. 
louisianabelieves.com/resources/library/test-results (accessed October 14, 2014). 


pep a the 18 New Orleans RSD schools that experienced what 
the district calls a full charter takeover between fall 2008 and fall 2013. Full take- 
overs convert all grades in the legacy school at the same time; the takeover school 
grandfathers legacy students in the relevant grades, and typically opens in the leg- 
acy school building. Alternatives to the full takeover model include principal-led 
conversions, phased-in takeovers, and school mergers. We focus on full takeovers 
because these are broad and well-defined transformations, with a clearly identified 
grandfathering cohort at the relevant legacy school. Our takeover analysis omits 
charter-to-charter takeovers, for which we were unable to construct a credible con- 
trol group (though these play a role in a supplementary analysis that allows for 
non-takeover charter effects). The two high schools in the table are also omitted; 
our analysis focuses on schools with middle school grades (in RSD, these are almost 
all K-8 schools) because this is where takeovers are most common and because the 
legacy school scores used in our IV strategy are unavailable for high schoolers. 
The decision to effect a full takeover at a low-performing RSD school was driven 
in part by average test scores and in part by the availability of an interested and 
acceptable charter operator. Operators typically applied for a charter early in the 
legacy year, with some indicating a preference for specific schools. Takeover deci- 
sions were usually announced no earlier than December of the legacy year, with 
the charter operator selected between January and May. Low test scores figured 
importantly in takeover decisions, but legacy schools have not usually been the very 


© Louisiana issues five types of charters, according to whether the charter is authorized by the local school board 
or the LDE, whether the school is new or a conversion, and whether it’s in RSD. RSD’s Type 5 charter schools, the 
focus of our study, are authorized and overseen by the LDE. 
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lowest-performers in the district. The matching strategy detailed below exploits the 
fact that many similarly low-performing direct-run schools continued to operate 
alongside legacy schools after the latter were converted to charters. 

Table Al shows that the 11 legacy schools in our study were taken over by 6 char- 
ter management organizations (CMOs), with the Crescent City and RENEW CMOs 
each operating multiple schools. In two 2013 takeovers, two legacy schools were 
merged into a single takeover school. Table Al also shows that seven out of nine 
study takeovers were operated by CMOs that identify with No Excuses pedagogy. 
The No Excuses model for urban education—sometimes also called “high expecta- 
tions”—is characterized by extensive use of tutoring and targeted remedial support, 
reliance on data and teacher feedback, a focus on basic skills, high expectations from 
students and staff, and an emphasis on discipline and comportment. New Orleans 
Parents’ Guide school brochures suggest that almost all takeovers enacted policies 
associated with No Excuses, including an extended school day, student uniforms, 
and parent involvement groups; many also extended the school year and added 
weekend classes. Angrist, Pathak, and Walters (2013) and Dobbie and Fryer (2013) 
present evidence suggesting that No Excuses practices explain the success of urban 
charters in Massachusetts and New York. 

RSD’s charter schools function outside the collective bargaining agreement 
between OPSB and the United Teachers of New Orleans union that represents teach- 
ers at non-charter OPSB schools and a few OPSB charters (Cowen Institute 2011). 
Appendix} Table A2 \compares teacher characteristics, expenditure, and class size 
at RSD direct-run and charter schools. Teachers at RSD charters tend to have less 
experience and earn lower base salaries than those at direct-run schools. Class sizes 
at takeover and legacy schools are similar and close to those seen at other charter and 
direct-run public schools. Per-pupil expenditure is somewhat lower at RSD charter 
schools, though this may reflect differences in the student body and the teacher 
experience distribution. The per-pupil expenditure contrast between takeover and 
legacy schools shows only a small gap. 


C. UP from Gavin Middle School 


We supplement the RSD analysis with estimates of attendance effects at UP 
Academy, Boston’s first in-district charter middle school. The UP Education 
Network is rapidly expanding, having recently assumed responsibility for manage- 
ment of two schools in Boston’s Dorchester neighborhood (one elementary and one 
K-8), and opened two (non-charter) middle schools in Lawrence, Massachusetts. 
Our middle school focus necessarily excludes BGA, Boston’s in-district charter 
high school. In this context, it’s worth noting that BGA is more of an in-district 
conversion than a charter takeover, since it was initially staffed by BPS teachers and 
administrators previously employed elsewhere in the district.8 


7 Table B1 in the online Appendix lists sources for this classification. 
8 Concerns about low achievement and other problems led the state to put BGA on probation in October 2014. 
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Boston’s in-district model is one of a number of policy experiments initiated at 
low-achieving schools in 2010.\As in RSD, the birth of an in-district charter reflects 
both the district’s desire to address poor performance and the interest expressed by 
a qualified charter operator: UP Education Network was selected partly because 
UP was ready to grandfather all Gavin students (Toness 2010). Gavin students 
were automatically admitted to UP Boston, though a simple application form was 
required (UP staff visited Gavin students’ homes to encourage application) (4 

Unlike other charter schools in Boston, which operate as independent districts 
and are funded by inter-district transfers, UP’s spending appears in the BPS budget. 
Former Gavin teachers were free to apply for positions at UP, and a handful did so, 
but their positions were not grandfathered and, according to school officials, none 
were ultimately hired. UP administrators and staff are part of the collective bar- 
gaining units representing other BPS workers, but the school functions in a looser 
framework established in memoranda between UP and the district. UP is required to 
pay collectively bargained salaries, but school leaders and UP administrators make 
personnel decisions freely, as in a nonunion workplace. 

As can be seen in column 8 of Table A2, which also compares teacher character- 
istics at the Boston schools in our study, UP’s teachers are much younger than the 
Gavin staff they replaced: 60 percent of the UP teachers in our sample are no older 
than 28. This is unusually youthful even by the standards of Boston’s other charter 
schools. UP class sizes are smaller and UP’s per-pupil expenditure is somewhat 
lower than at the Gavin school. Like most of our RSD takeovers, the UP charter 
aligns the school with No Excuses principles" The UP school day is two hours 
longer than the Gavin day had been and UP teachers are expected to report for work 
on August 1. 


D. Related Research on Takeovers and Turnarounds 


Dee (2012) uses the test proficiency cutoffs that determine SIG qualification 
to evaluate SIG participation in a regression discontinuity design. Dee’s estimates 
suggest that SIG-funded interventions improve performance for students at treated 
schools. A companion difference-in-differences analysis points to the intermediate 
federal turnaround model as the most effective, while estimates for the remaining two 
SIG strategies, including restarts, are not significantly different from zero. It’s worth 


°Gavin and Odyssey were among BPS’s lowest-performing schools in 2010, though not the lowest. The 
state categorized these schools as “Level 3,’ meaning they were found in the bottom 20 percent of the relevant 
grade-specific performance distribution. In response to our queries, BPS administrators emphasized that in-district 
charter conversion was one of several strategies available to the district for schools in Level 3. Lower-ranked 
Level 4 schools were not eligible for in-district conversion. 

'0Some high needs special education students at Gavin were grandfathered into the Richard Murphy school, 
which operates a satellite program in the former Gavin building (BPS 2013, p. 6, p. 146). These cases notwithstand- 
ing, the overall UP enrollment take-up rate for grandfathered special education students is close to that for other 
grandfathered students. Estimates conditional on baseline special education status are also similar to those from 
the full sample. 

'l Specifically, UP’s charter application states “all stakeholders should not make or accept excuses for any- 
thing less than excellence,” and describes key No Excuses practices as part of their educational programming 
(UP Academy 2010). More recent school documents emphasize a culture of “high expectations.” (http://www. 
upeducationnetwork.org/uploads/documents/15-1015-UPEN-frequently-used-terms-vf.pdf, accessed May 5, 2016.) 
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noting, however, that very few California schools opted for the more radical restart 
intervention, and Dee’s (2012) estimates for the restart treatment are imprecise. 

Houston’s pioneering Apollo 20 program revamped educational practices along 
No Excuses’ lines in 20 of Houston’s lowest performing schools, while also replac- 
ing most school leaders and half of the teaching staff in these schools; a similar 
effort was undertaken on a smaller scale in Denver. The insertion of charter school 
best practices in existing public schools provides a natural alternative to the takeover 
model studied here, and qualifies for the same sort of federal support. Fryer’s (2014) 
analysis of Apollo using randomized and quasi-experimental research designs 
shows statistically significant gains in math of between one-fifth and one-sixth of a 
standard deviation, with little effect on reading. In the spirit of our grandfathering 
strategy, Fryer’s quasi-experimental analysis uses baseline enrollment zones to con- 
struct instruments. !* 

CREDO (2013b) evaluates the effects of attending three RSD takeover charters. 
The CREDO study contrasts students based on baseline and post-takeover enroll- 
ment status, comparing, for example, students who move into and who exit from 
schools slated for charter conversion. The potential for selection bias would seem 
to make these sorts of comparisons hard to interpret. In related work, CREDO 
(2013c) reports modest gains from the New Orleans charter sector as a whole in 
a national matched-pair study of overall urban charter school effectiveness. Along 
the same lines, McEachin, Welsh, and Brewer (2014) offer a regression-controlled 
value-added style analysis of New Orleans school sectors that does not isolate 
effects on takeover students. 


II. Grandfathering Identification 
A. The RSD Comparison Group 


Our grandfathering research design uses a combination of matching and regres- 
sion to mitigate omitted-variables bias in comparisons of grandfathering-eligible 
and ineligible students. To see how the matched comparison group is constructed, 
consider the set of sixth graders enrolling at an RSD school slated for takeover 
at year’s end: sixth grade legacy school enrollment entitles this group to seventh 
grade seats in the takeover charter. Since legacy and takeover schools in RSD typ- 
ically enroll grades K—8, there are few non-legacy sixth graders who share a fifth 
grade school with the grandfathering-eligible group. We therefore look for a com- 
parison group in the population of sixth graders not enrolled at the legacy school, 
but who attended schools similar to those attended by legacy school students in 
fifth grade (we refer to these as baseline schools). Specifically, baseline schools 
are considered matched when they have School Performance Scores (SPS) in the 


'2 Unlike Fryer (2014), our grandfathering strategy matches on baseline school characteristics to eliminate 
covariate differences associated with the grandfathering instrument and allows for violations of the exclusion 
restriction that may compromise naive matched comparisons. In a methodologically related study, Jacob (2004) 
also uses an initial condition—whether a student resides in a public housing building later slated for demolition—as 
an instrument for the effect of public housing on children’s achievement. 
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TABLE 1—GRADE PROGRESSION IN THE GRANDFATHERING RESEARCH DESIGN 


Legacy 
Baseline Legacy enrollment years 
grade — grade First Second Third Fourth (No. of schools) 


Takeover grades 


Panel A. RSD 


Study takeovers 3 4 5 6 7 8 2009-2010 (5) 
4 5 6 of 8 2010-2011 (1 
5 6 7 8 2011-2012 (1 
6 7 8 2012-2013 (4 

Panel B. Boston 

UP > 6 i 8 2010-2011 (1 
5 i 8 

Dearborn/Harbor 5 6 7 8 2009-2010 (2 
5 7 8 

Orchard Gardens 5 6 7 8 2009-2010 (1 
6 7 8 


Notes: This table summarizes grade-based timing for the selection of baseline schools, 
grandfathering eligibility, and takeover outcomes in the RSD and Boston analysis samples. 
Grandfathering eligibility is determined by enrollment in the fall of the legacy enrollment year, 
while matching uses information from the baseline grade. Outcomes are from the spring of the 
corresponding school year for each takeover grade. The number of legacy schools in each aca- 
demic year appears in parentheses. 


same five-point bin! In addition to baseline schools, the RSD comparison sample 
matches grandfathering-eligible and ineligible students on race, sex, baseline year, 
baseline special education status, and baseline subsidized lunch eligibility. 

In practice, the RSD grandfathering experiment involves multiple grades, 
schools, and years. The relationship between legacy grades, baseline grades, and 
takeover grades is described in|Table 1| Because the earliest available baseline infor- 
mation is from third grade, the RSD sample covers legacy school enrollment in 
grades four through seven and takeover charter enrollment in grades five through 
eight. Potential takeover exposure thus ranges from one year for students in sev- 
enth grade in the legacy year to four years for students in fourth grade in the legacy 
year (or more if grades are repeated). Students may have been eligible for grand- 
fathering into multiple takeover charters; the grandfathering instrument indicates 
eligibility at any of the takeover schools we study. When pooling across grades, we 
retain students in the first year they become grandfathering-eligible or are matched 
to a grandfathering-eligible student. The number of grandfathering-eligible students 
enrolled in a legacy school in the fall of the year prior to takeover averages roughly 
70 students per school and is about one-third the size of the matched comparison 
group (Table B2 in the online Appendix reports the number of observations contrib- 
uted by each RSD legacy school). 

The primary RSD outcomes are math and English Language Arts (ELA) scores 
from the Louisiana Educational Assessment Program (LEAP) in fourth and eighth 
grade and the Integrated Louisiana Educational Assessment Program (iLEAP) in 


'3 SPS scores are used for accountability purposes within RSD. In the period relevant to our study, SPS scores 
ranged from 0 to 200. Our results are virtually unchanged when smaller bins are used; bins wider than about 10 
points generate a coarse match with many low-scoring schools grouped together. Instrument balance, documented 
in Table 2 and discussed below, is driven mainly by matching on SPS bins. 


1888 THE AMERICAN ECONOMIC REVIEW JULY 2016 


grades five through seven. Scores are from spring 2011, the first post-takeover test 
date for the schools in our sample, through 2014/4 A data Appendix details the 
construction of our analysis files from raw student enrollment, demographic, and 
outcome data. For the purposes of statistical analyses, scores are standardized to the 
population of RSD test-takers in the relevant subject, grade, and year. 

As can be seen in the first two columns of almost all RSD and RSD 
charter-bound students (those enrolled in an RSD charter school in the grades fol- 
lowing baseline) are black, and most are poor enough to qualify for a subsidized 
lunch. RSD charter-bound students have baseline scores near the overall district 
mean (zero, by construction). By contrast, students who enroll in takeover char- 
ters and those eligible for grandfathering have much lower baseline test scores. For 
example, the average baseline math score of grandfathering-eligible students in our 
analysis sample is around 0.27¢ below the corresponding RSD population average. 
This marks an important contrast with samples of lottery applicants at oversub- 
scribed charter schools, a group that’s often positively selected on baseline char- 
acteristics,!> Columns 3-5 of Table 2 compare characteristics of takeover charter 
students and grandfathering-eligible students with those of the grandfathering com- 
pliers for whom grandfathering instruments identify average causal effects. The lat- 
ter group is defined as the set of students who enroll in an in-district charter when 
grandfathered but not otherwise!4 Compliers’ baseline scores are not as low as 
the scores in the population of students at risk for grandfathering, but they still fall 
around 0.1—0.15c0 below the district average. 

The RSD comparison group appears to be well-matched to the RSD 
grandfathering-eligible sample. This is documented in column 6 of Table 2, which 
reports regression-adjusted differences in variables that were not used for matching. 
The balance coefficients in column 6 come from a model that includes a full set of 
matching-cell fixed effects, with no further controls. These estimates show no sta- 
tistically significant differences in limited English proficiency rates or in baseline 
scores. 

Table B3 in the online Appendix reports follow-up rates and measures of dif- 
ferential attrition in the RSD analysis sample. Follow-up scores are available for 
almost three-quarters of students in the first two post-takeover years. The follow-up 
rate declines in years three and four, reflecting RSD’s highly mobile low-income 
population. Importantly, however, the likelihood an RSD student contributes an out- 
come score to the analysis sample is unrelated to his or her grandfathering eligibility 
within matching cells. Column 7 of Table 2 similarly shows that baseline covariates 
are balanced in the subsample for which we can measure outcomes. 


‘47 EAP and iLEAP include multiple-choice and open-answer questions. LEAP scores are used for determining 
grade progression according to Louisiana state guidelines. The iLEAP test combines a test of academic standards 
and (through 2013) a norm-referenced component from the Iowa Test of Basic Skills (ITBS) through 2012-2013. 

'STn the middle school sample analyzed in Abdulkadiroglu et al. (2011), for example, the baseline math gap 
between charter applicants and Boston students is around 0.360. 

'©Following Abadie (2003), complier means are computed as weighted averages, weighting by 

1 D(1 — Z) (1-D)Z 
1—E[Z|X]  E|Z|X) 
grandfathering eligibility. For this purpose, E|Z|X] is estimated by a saturated regression of the grandfathering 
instrument on matching-cell fixed effects. 


, where D denotes takeover enrollment in the first exposure year and Z denotes 
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TABLE 2—RSD DEscRIPTIVE STATISTICS AND GRANDFATHERING BALANCE 


Sample means 


RSD Analysis sample Balance coefficients 
Charter- Takeover Grandfathering- First 
All bound charter eligible Grandfathering Analysis exposure 
students students students students compliers sample year 
() (2) (3) (4) (5) (6) (7) 
Hispanic 0.026 0.024 0.018 0.029 0.008 —_— — 
Black 0.964 0.971 0.994 0.982 0.997 — — 
White 0.019 0.016 0.008 0.016 0.004 — — 
Asian 0.008 0.008 0.001 0.009 0.000 —_— — 
Female 0.475 0.473 0.489 0.501 0.496 — — 
Special 0.069 0.066 0.071 0.093 0.048 — — 
education 
Free/reduced 0.912 0.926 0.955 0.919 0.963 — — 
price lunch 
Limited English 0.017 0.016 0.013 0.020 0.007 0.000  —0.001 
proficient (0.001) (0.001) 
N 14,575 11,381 1,040 763 2,061 3,503 2,572 
Baseline math —0.001 0.019 —0.320 —0.266 —0.151 —0.019 —0.042 
score (0.048) (0.052) 
N 12,960 10,565 1,038 760 2,059 3,500 2,570 
Baseline math —0.099 —0.084 —0.261 —0.254 —0.281 0.007 0.009 
gain (0.069) (0.081) 
N 4,871 4,099 330 241 819 1,235 993 
Baseline ELA 0.000 0.022 —0.303 —0.261 —0.112 —0.009 —0.032 
score (0.048) — (0.055) 
N 12,967 10,572 1,040 762 2,061 3,502 2,572 
Baseline ELA —0.105 —0.097 —0.181 —0.182 —0.141 —0.015 0.001 
gain (0.072) (0.079) 
N 4,879 4,105 330 241 819 1,235 993 


Notes: This table reports sample means and coefficients from regressions of the variable in each row on a grandfa- 
thering eligibility dummy indicating enrollment in an RSD takeover legacy school in the fall of the academic year 
prior to takeover. Baseline test score gains are relative to the previous pre-baseline grade, constructed only for the 
subsample with pre-baseline information. All regressions include matching cell fixed effects (cells are defined by 
race, sex, special education status, subsidized lunch eligibility, baseline grade and year, and baseline school SPS 
scores in five-point bins). The sample in columns 3-7 is restricted to students enrolled in an RSD direct-run school 
at baseline. Column | reports means for a sample of RSD students enrolled in the same baseline years as the anal- 
ysis sample, while column 2 is restricted to those students that enroll in an RSD charter school in grades follow- 
ing the baseline grade. Column 3 reports means for students that enroll in a takeover charter in potential takeover 
grades, while column 4 describes students enrolled in a legacy school. Column 5 reports means and counts of grand- 
fathering compliers. Robust standard errors are reported in parentheses. 


B. RSD Grandfathering Graphics 


We motivate the grandfathering identification strategy for RSD with a graphical 
comparison of achievement trends in the grandfathering-eligible and matched com- 
parison samples. Provided scores in the eligible cohort and the comparison group 
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FiGurE 2. Test SCORES IN THE RSD GRANDFATHERING SAMPLE 


Notes: Panel A plots average LEAP/iLEAP math and ELA scores of students in the RSD leg- 
acy middle school matched sample. Panel B plots achievement growth relative to the baseline 
(—1) grade. Estimates in both panels control for matching cell fixed effects. Scores are stan- 
dardized to those of students at direct-run schools in New Orleans RSD, by grade and year. 
Grade 0 is the last grade of legacy school enrollment. 


move in parallel in the pre-takeover period, differences in score growth between 
eligible and ineligible students in the post-takeover period offer compelling evi- 
dence of a takeover treatment effect. The scores plotted here are standardized to 
samples of students at RSD’s direct-run schools, so achievement trends are cast 
relative to this group (the statistical analysis uses scores standardized to the district, 
as in Table 2). 

The upper panels of|Figure 2 khow remarkably similar pre-takeover trajectories 
for the math and ELA scores of grandfathering-eligible students and their matches 
(as for the balance regression estimates reported in Table 2, Figure 2 compares 
residuals from a regression on matching-cell fixed effects with no other controls). 
Consistent with RSD’s goal of transforming low-performing schools, relative 
achievement in the grandfathering-eligible group declines in the grade before take- 
over. Importantly, the pre-takeover dip (reminiscent of the pre-treatment earnings 
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dip documented by Ashenfelter (1978) for applicants to job training programs) is 
mirrored in the matched comparison group. The comparison in Figure 2 does not 
adjust for baseline student achievement, so parallel trends are not guaranteed, but 
rather reflects the success of the matching strategy in producing similar treatment 
and control groups. 

Matching effectively eliminates baseline differences by grandfathering status, so 
simple post-treatment comparisons seem likely to reveal causal effects. Difference- 
in-differences (DD) style comparisons of achievement growth appear in the lower 
panel of Figure 2, which plots achievement growth in the grandfathering-eligible and 
ineligible subsamples relative to the baseline grade. Pre-baseline growth differences 
by grandfathering status are centered around zero, while achievement contrasts after 
the legacy year strongly favor the grandfathered cohort. Since around 66 percent of 
students are caused to matriculate at a takeover charter when grandfathered (a figure 
reported in Table 3, panel C), this pattern suggests takeover enrollment significantly 
boosts achievement. 

Figure 2 shows parallel pre-takeover trends in years up to, but not including, 
the last grade of legacy school enrollment (grade 0 in the figure). The negative and 
significant (for math) DD contrast in the last legacy grade signals a possible causal 
effect of legacy enrollment per se, regardless of whether legacy attendance leads to 
subsequent enrollment in the takeover charter. This is an unsurprising but poten- 
tially important finding: legacy schools were slated for closure in part because of 
extraordinarily low and even declining achievement. Moreover, closure itself might 
be disruptive, with lasting consequences for legacy students. Our grandfathering [V 
strategy therefore allows for direct effects of legacy school attendance when using 
legacy school enrollment to instrument takeover attendance. 


C. Econometric Framework 


Consider a group of legacy school students and their matched comparison coun- 
terparts with covariate values falling in a single matching stratum. Achievement 
for each student is observed in two grades: at the end of the legacy grade, imme- 
diately prior to the takeover (grade /), and after the takeover (grade g). The 
grandfathering-eligible group is mostly enrolled in the takeover school in grade g, 
while few in the comparison group are. A dummy variable Z—the grandfathering 
instrument—indicates legacy school enrollment in grade / (observed at the start of 
the school year) while the variable D indicates takeover school enrollment at any 
time in grade g. Achievement in the two grades is denoted Y’ and Y8, observed at the 
conclusion of the school year. 

Legacy school enrollment in grade / potentially affects grade g achievement 
through two channels: by increasing the likelihood of takeover attendance in grade g 
and by adding a year’s exposure to the legacy school in grade /, an event that may have 
lasting consequences. Potential outcomes in grade g are therefore double-indexed. 
Specifically, we write YS, to indicate the grade g outcome that would be observed 
when Z = z and when D = d. Potential outcomes in grade /, written ¥, are indexed 
against Z alone, since grade / predates takeover exposure. Using the potential treat- 
ments notation of Imbens and Angrist (1994), legacy enrollment shifts takeover 
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exposure from Dp to D,. In this setup, observed outcomes are determined by poten- 
tial outcomes and grandfathering as follows: 


Y= 27). 
D = Dy + Z(D, — Dy). 


¥? = Vf + Z(¥f — Yoo) + D( V6 — Yio + Z(¥fi — Yfo — (Yi — Ybo))) 
= Yoo + Z(Yio — Yoo) 
+ (Dy + Z(D Dy)) (Vi as Yoo+Z (Vii — Yio — (Yor - Yio)) ) 


where the last line uses the expression for D to obtain a representation for observed 
Y® as a function of potential outcomes, potential treatments, and the instrument. 


We assume potential outcomes and treatments satisfy the following assumptions: 
ASSUMPTION 1 (Independence): (Y), Y1, Yo, Yé1. Yfos ti, Do, Di) LL Z. 
ASSUMPTION 2 (Monotonicity): Pr(D, > Do) = 1. 

ASSUMPTION 3 (First stage): Pr(D; > Dy) > 0. 


Assumption 1—Independence—asserts that the grandfathering instrument is as 
good as randomly assigned, that is, independent of potential outcomes and poten- 
tial treatment take-up (implicitly, within matching strata). Table 2 and Figure 2, 
which show that matching eliminates covariate and baseline score differences in 
our RSD analysis sample, support this. Monotonicity says that legacy enrollment 
either induces takeover enrollment or has no effect for everyone in the analysis 
sample. Assumption 3 requires legacy enrollment to induce takeover enrollment, 
at least for some. 

As in the Angrist, Imbens, and Rubin (1996) framework for identification of 
local average treatment effects (LATEs) with possible violations of the exclusion 
restriction, Assumptions 1-3 allow for direct effects of legacy exposure on grade g 
outcomes. Such effects arise if Y/; 4 Y$q when D is fixed at d. In other words, main- 
taining the assumption that legacy enrollment is as good as randomly assigned, 
we’ve allowed for violations of the exclusion restriction associated with use of Z as 
an instrument for D. This is motivated by the possibility that an additional year of 
exposure to a low-performing school has lasting effects. 

Rather than defend a conventional exclusion restriction in this setting, we 
replace it with a weaker restriction on potential achievement gains. This allows for 
direct additive effects of legacy enrollment that are free to vary within the LATE 
subpopulations of always-takers (those with D,; = Dy = 1), never-takers (those 
with D, = Dy = 0), and compliers (those with D, > Dp): 


ASSUMPTION 4 (Gains Exclusion): E[Y{,— Y{|T] = E[¥$a— Y4|T|. where 
T= aDy + n(1 — D,) + c(D, — Dp) identifies always-takers (a), never-takers (n), 
and compliers (c). 
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Assumption 4 requires that expected potential achievement gains be the same for 
those who do and don’t attend the legacy school in grade /, once takeover enrollment 
is fixed. This allows for Y{, Yoq, while also weakening the canonical exclusion 
restriction applied to gains, which says that Yj, — Y! = Ys, — Yé for everyone rather 
than just on average. Balance in pre-baseline to baseline score gains by grandfather- 
ing eligibility status—documented for the RSD matched sample in Table 2—serves 
as an indirect test of this assumption. 

Assumption 4 is justified by an additive structure for expected potential outcomes 
in each grade: 


(1) E[¥i|T = s] = ay, + 27%, 


(2) E|YalT 


s| = os + 2Y5 + dp,. 


The parameters a , and a», in these expressions are subgroup-specific potential 
outcome means with both the legacy- and takeover-enrollment indicators switched 
off; y, is an additive legacy school enrollment effect common to grades / and g, 
and (3, is the causal effect of takeover attendance for LATE subgroup s. This additive 
model rules out interactions between legacy and takeover attendance effects, while 
allowing legacy effects to persist across grades. 

The theorem below (proved in the Appendix) shows that under Assumptions 1-4, 
a Wald-type IV estimator applied to achievement gains captures the average causal 
effects of takeover attendance on grandfathering compliers’ achievement: 


THEOREM 1: Under Assumptions 1-4, 


Ely’—y'|z = 1|-e£[y8-y'|z = 0 
E[D|Z = 1] E|D|Z = 0) 


= EY, — Yfo|D, > Do] 


E|Y¥§: — Yéo|D; > Dal. 


In the notation of equations (1) and (2), this theorem establishes identification of (, 
for a model where legacy enrollment has direct effects. 

We use Theorem | in two ways: to capture causal effects of Bernoulli takeover 
enrollment in the year following a takeover and to capture causal effects of years of 
takeover exposure on outcomes in later years. The latter is supported by an exten- 
sion of Theorem 1 detailed in the Appendix, which shows how the IV estimand 
for an ordered treatment can be interpreted as a convex combination of incremen- 
tal average causal effects. The Appendix also discusses results from a model that 
relaxes Assumption 4. 

These econometric considerations motivate a two-stage least squares (2SLS) 
estimator with second-stage estimating equation that can be written 


(3) Yi—Y} = a’X, + 3 Kjdy + BD + Nits 
j 


where Y% is student i’s score in year t in grade g and Y! is i’s score in the last grade in 
which he or she was potentially enrolled at the legacy school. The treatment variable 
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here, D,,, counts the number of years student i spent at the takeover school as of 
year t, up to and including the grade enrolled in that year (D,, is Bernoulli for tests 
taken in the first year of takeover operation). 

The first stage equation that accompanies (3) is 


(4) Dj = 0X, + 2 Hjdy + 1Z; + Viz, 
J 


where Z; is the excluded instrument, an indicator of legacy enrollment in the fall 
of the legacy school’s final year in operation, and 7 is the associated first stage 
coefficient. As with the models used to investigate covariate balance, equations (3) 
and (4) control for matching cell fixed effects. In particular, because the compar- 
ison sample is generated by an exact match on race, sex, baseline special educa- 
tion status, baseline subsidized lunch eligibility, baseline school SPS bins, baseline 
year, and the legacy grade, equations (3) and (4) include dummies for each of these 
cells, denoted d; for cell j. The empirical first- and second-stage models also include 
dummies for English proficiency and year-of-test (denoted by the vector X;,, with 
coefficients a and 6). Finally, although baseline scores appear to be uncorrelated 
with grandfathering exposure in RSD, X;, includes these controls to boost precision. 


III. Charters without Lotteries in New Orleans RSD 
A. Grandfathering Results 


Each year of enrollment in an RSD takeover charter increases math and ELA 
scores by an average of 0.210 and 0.140, respectively (the associated standard errors 
are on the order of 0.04). These IV estimates, reported in the last column off Table 3| 
are generated by a first stage of about 1.1 years of additional takeover exposure fe 
grandfathered students (first stage estimates are reported in column 3 of the table) fr 

Analyses that disaggregate by outcome grade and by years of potential takeover 
exposure show that takeover effects are larger in seventh and eighth grade than ear- 
lier, and are larger in the first two years of takeover exposure than later. The first 
stage effect of grandfathering eligibility on enrollment in the first exposure year, 
reported at the top of panel C, reveals that grandfathering boosted initial takeover 
enrollment rates by around 66 percentage points. 

The IV estimates generated by the grandfathering design exceed (and, in many 
cases, are significantly different from) the corresponding OLS estimates reported 
in column 2 of Table 3. This suggests that uninstrumented comparisons by take- 
over enrollment status, such as those reported in CREDO (2013b), may be biased 
downward. 

IV estimates that fail to adjust for possible effects of pre-takeover legacy school 
enrollment also appear to be too small. Fitting versions of equations (3) and (4) to 
post-treatment levels rather than gains generates math and ELA effects of 0.160 


'7Tn January 2016, RENEW SciTech Academy was accused of breaking state testing rules in the 2014-2015 
school year (Dreilinger 2016). This year is not in our data. Estimates of takeover effects are changed little by the 
omission of SciTech and ReNEW schools. 
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TABLE 3—GRANDFATHERING IV ESTIMATES OF RSD TAKEOVER EFFECTS 


2SLS 
Comparison Enrollment 
group mean OLS First stage effect 
Q) (2) (3) (4) 
Panel A. All grades 
(Fifth through eighth) Math (N = 5,625 —0.089 0.123 1.073 0.212 
(0.020 (0.052) (0.038 
ELA (N = 5,621 —0.092 0.082 1.075 0.143 
(0.018 (0.052) (0.039 
Panel B. By grade 
Fifth and sixth grades Math (N = 2,579 —0.091 0.099 0.738 0.165 
(0.035 (0.041 (0.068 
ELA (N = 2,579 —0.116 0.023 0.745 0.101 
(0.033 (0.042) (0.070 
Seventh and eighth grades Math (N = 3,046 —0.086 0.133 1.355 0.231 
(0.020 (0.070) (0.037 
ELA (N = 3,042 —0.071 0.104 1,352 0.171 
(0.019 (0.070 (0.036 
Panel C. By potential exposure 
First exposure year Math (N = 2,553 —0.105 0.200 0.659 0.230 
(fifth through eighth grades) (0.044 (0.023) (0.069 
ELA (N = 2,553 —0.103 0.099 0.659 0.197 
(0.043 (0.023 (0.068 
Second exposure year Math (N = 1,664 —0.151 0.168 1.148 0.332 
(sixth through eighth grades) (0.031 (0.061 (0.058 
ELA (N = 1,664 —0.124 0.101 1.158 0.158 
(0.028 (0.061 (0.051 
Third and fourth exposure year Math (N = 1,408) 0.015 0.097 1.698 0.117 
(seventh and eighth grades) (0.022 (0.131 (0.042 
ELA (N = 1,404) —0.033 0.077 1.698 0.094 
(0.020 (0.132) (0.043 


Notes: This table reports OLS and 2SLS estimates of the effects of RSD takeover charter enrollment on fifth through 
eighth grade LEAP/iLEAP math and ELA test scores using the grandfathering eligibility instrument. The sample in 
columns 2-4 includes RSD direct-run school students matched to legacy school students. The endogenous regres- 
sor counts the number of years enrolled at a takeover charter prior to testing. All models control for matching strata, 
limited English proficiency, baseline test scores, and year/grade effects. Robust standard errors, clustered by stu- 
dent, are reported in parentheses. Means in column | are for grandfathering-ineligible matched students. 


and 0.11o, respectively. Differences between these estimates and those for gains 
are consistent with the negative legacy-year treatment effects suggested by Figure 
2. Appendix|Table A3 feports legacy year treatment effects and estimates of models 
that weaken Assumption 4 to allow for partial pass-through of legacy effects—these 
estimates differ little from the estimates reported in Table 3/5 


B. Interpreting RSD Takeover Effects 
The RSD grandfathering identification strategy compares students that mostly 


attend takeover charters with a grandfathering-ineligible comparison group that 
went to various sorts of schools. Most students in the comparison group began 


'8 Estimates of effects on science and social science are reported in Table B4 of the online Appendix. Estimates 
by sex and baseline achievement appear in online Appendix Table BS. 
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TABLE 4—THE CHANGING SCHOOL SECTOR DISTRIBUTION IN RSD 


All students Compliers 
Z=0 Z=1 Z=0 Z=1 
() (2) (3) (4) 
Panel A. First exposure year 
Study takeover 0.089 0.780 0.000 1.000 
Other RSD takeover 0.072 0.032 0.087 — 
Non-takeover RSD charter 0.331 0.154 0.316 —_ 
RSD direct-run 0.508 0.034 0.597 a 
N 2,027 531 
Panel B. Second exposure year 
Study takeover 0.206 0.714 0.000 1.000 
Other RSD takeover 0.105 0.016 0.142 — 
Non-takeover RSD charter 0.395 0.223 0.393 —_ 
RSD direct-run 0.294 0.047 0.465 — 
N 1,349 318 
Panel C. Third exposure year 
Study takeover 0.277 0.754 0.000 1.000 
Other RSD takeover 0.112 0.031 0.218 — 
Non-takeover RSD charter 0.450 0.188 0.517 —_ 
RSD direct-run 0.161 0.026 0.265 — 
N 795 191 
Panel D. Fourth exposure year 
Study takeover 0.316 0.646 0.000 1.000 
Other RSD takeover 0.167 0.051 0.030 — 
Non-takeover RSD charter 0.485 0.291 0.864 —_— 
RSD direct-run 0.032 0.013 0.106 — 
N 342 79 


Notes: Columns | and 2 contrast the school sector distribution for grandfathering eligible 
(Z = 1) and ineligible (Z = 0) students; columns 3 and 4 show the same for grandfathering 
compliers. The category labeled “other RSD takeover” includes charter-to-charter conversions, 
principal-led conversions, and mergers. Non-takeover RSD charters include startup charters 
created since the 2008-2009 academic year, and charters operating as of 2007-2008. 


middle school at one of RSD’s direct-run public schools. But the distribution of 
takeover alternatives evolved as RSD closed its direct-run schools and as students 
changed schools for reasons other than closure. Estimates of RSD takeover effects 
therefore reflect a growing share of charter-to-charter comparisons. If non-takeover 
charters also boost achievement, the takeover effects reported in Table 3 might mask 
a higher charter-versus-traditional average causal effect. 

[Table 4 |describes the counterfactual school sector distribution in detail, focus- 
ing on the distinction between the charters that define the takeover treatment for 
the purposes of Table 3 (“study takeovers”), other takeover schools (including 
charter-to-charter conversions), non-takeover RSD charters, and direct-run RSD 
schools. Specifically, the first two columns show the distribution of school types 
by grandfathering status, while column 3 describes the types of schools attended by 
untreated compliers. Complier counterfactuals are constructed by estimating causal 
effects of the takeover enrollment dummy, D, on a vector of school sector indicators, 
W. Associated with each W are potential attendance outcomes, Wo and W,, describ- 
ing school choices in non-treated and treated states (that is, potential school type 
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when D = 0 and D = 1). Column 3 of Table 4 reports estimates of E |W |D; > Dol, 
the distribution of school types among compliers when they don’t enroll in a take- 
over]! By definition, treated compliers enroll in a takeover school when they’re 
grandfathering-eligible; column 4 in the table is included as a reminder of this fact. 

The grandfathering first stage contrasts a 78 percent first-year takeover enroll- 
ment rate for those grandfathered (reported in column 2 of Table 4) with a 9 percent 
comparison group enrollment rate (reported in column 1). The first-year increase 
in study takeover enrollment reflects a substantial reduction in attendance at 
non-takeover charters (compare 0.33 with 0.15) and, especially, a sharp reduction 
in attendance at direct-run schools (compare 0.51 with 0.03). The counterfactual 
attendance distribution in column 3 shows that 32 percent of untreated compliers 
enrolled initially in a non-takeover charter school, while 60 percent attended a 
direct-run school. 

Not surprisingly, both the takeover first stage and the proportion of the 
non-grandfathered comparison group enrolled in direct-run schools shrank over our 
sample period. The (study) takeover first stage in the third year of potential takeover 
exposure was around 0.48 (0.754—0.277), while the counterfactual direct-run enroll- 
ment share for compliers fell to about 0.27. The remainder of third-year non-treated 
complier enrollment was in other RSD charter schools. Reflecting RSDs accelerat- 
ing charter transformation, the other-charter enrollment rate for compliers exceeded 
86 percent after four years of exposure. 

The growing share of the RSD comparison sample enrolled in charter schools 
dilutes estimated takeover effects if other charter schools generate similar achieve- 
ment gains. This observation motivates a 2SLS model with two endogenous 
variables, one tracking attendance at study charters and one tracking attendance at 
other charters. Our model with two charter treatments is 


(5) Yi —Y} = a’X;, + » Ky + BpDy + BoC + Nits 
j 


where C;, counts the number of years of attendance in charters other than those cov- 
ered by D,,. Equation (5) is identified here by the addition of interactions between the 
grandfathering instrument and covariates to the instrument list (specifically, 22 inter- 
actions with dummies for baseline year, baseline grade, and special education sta- 
tus/SPS bins). These interactions contribute to the first stage for CG, because students 
with differing characteristics are more or less likely to wind up in non-takeover 
charters in the event they aren’t grandfathered. This multiple-instruments strategy 
therefore identifies Gp and Gc in a constant-effects framework. 

As can be seen in the contrast between the estimated takeover effects reported 
in columns | and 2 of removing other charters from the counterfactual 
outcome distribution increases the estimated takeover effect on math scores by 


The counterfactual school type distribution is estimated using a weighting scheme sim- 
ilar to that used to construct complier characteristics means in Table 2. The weights in this case are given by 
E{Z|X] —Z 
o=(1 — D) 
E[Z| X](1 — E|Z|X}) 
as were used for Table 3, as well as matching cell fixed effects. Abdulkadiroglu, Angrist, and Pathak (2014) sim- 
ilarly estimate the counterfactual school sector distribution for applicants to Boston and New York exam schools. 


. E|Z| X] is modeled using a probit specification that includes the same controls 
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TABLE 5—GRANDFATHERING IV ESTIMATES OF RSD CHARTER EFFECTS 


() (2) (3) (4) (5) (6) 


Panel A. Math 


Takeover charter 0.210 0.358 0.344 
(0.037) (0.079) (0.138) 
[72.3] [33.4] [194.9] 
Other RSD charter 0.342 0.308 
(0.164) (0.376) 
[12.6] [27.4] 
Any RSD charter 0.366 0.359 0.385 
(0.060) (0.095) (0.071) 
[41.7] [237.9] [223.4] 
No. of instruments 23 23 2, 23 2 1 
N 5,625 5,625 2,553 5,625 2,953 5,625 
Panel B. ELA 
Takeover charter 0.141 0.154 0.205 
(0.038) (0.077) (0.140) 
[71.4] [33.8] [192.8] 
Other RSD charter 0.031 0.022 
(0.161) (0.363) 
[12.7] [27.1] 
Any RSD charter 0.214 0.281 0.257 
(0.061) (0.098) (0.072) 
[43.0] [238.0] [228.0] 
No. of instruments 23 23, 2 23 2 1 
N 5,621 5,621 2,553 5,621 23593 5,621 


Notes: This table reports 2SLS estimates of the effects of enrollment in study takeovers and 
other RSD charters. The endogenous regressors count the number of years enrolled in RSD 
charters prior to testing. The grandfathering instrument is used to construct the estimates in col- 
umn 6. Columns 1, 2, and 4 report 2SLS estimates from models that add interactions between 
grandfathering eligibility and baseline special education/SPS bin cells, baseline year, and 
baseline grade to the instrument list. The estimates in columns 3 and 5 add only interactions 
with a dummy for baseline year after 2009 and limit the sample to the first outcome grade. All 
models control for the covariates used to construct the estimates in Table 3. Robust standard 
errors, clustered by student, are reported in parentheses. First-stage F-statistics are reported 
in brackets (in models with two endogenous variables, these are as described in Angrist and 
Pischke 2008). 


70 percent. Column 1 reports an over-identified estimate of the takeover effect anal- 
ogous to the just-identified estimates of takeover effects reported in Table 3, while 
column 2 reports 2SLS estimates of 3, and 3-. The takeover estimate for math in 
the latter specification rises to 0.360, while the other RSD charter effect is a less 
precisely estimated 0.340. These results are similar to the estimates of math effects 
for Boston charter lottery applicants reported in Abdulkadiroglu et al. (2011), and 
much larger than the observational estimates for New Orleans charters found by 
CREDO (2013c). At the same time, the other-charter ELA effect in column 2 is 
close to zero. Consequently, the takeover effect on ELA scores remains near 0.140 
when estimated with or without a second endogenous variable. 

Although motivated by a constant-effects model, 2SLS estimates of Gp and Gc in 
equation (5) have a LATE interpretation when treatment is Bernoulli and average 
causal effects are mean-independent of the stratification variables used to generate 
instruments (Hull 2015). It is therefore of interest to consider models identified with 
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fewer interactions, for which the associated homogeneity assumptions are weaker. 
Column 3 of Table 5 shows estimates from the first outcome grade (so both D, 
and C;, are Bernoulli), computed with instrument interactions limited to an indica- 
tor for post-2009 baseline year. The homogeneous effects assumption here requires 
only charter effect stability across cohorts. Though imprecise, these estimates are 
close to those reported in column aP4 

The estimates in columns 2 and 3 of Table 5 suggest takeover and other RSD char- 
ters have similar effects on math scores. Assuming effects are the same in the two 
types of schools, we can estimate acommon charter effect by fitting a version of equa- 
tion (5) that replaces BpD;, + BcC, with 3,Aj,, where the variable A;, = D,, + C, 
counts years of attendance at any RSD charter. The resulting estimates of 3,, reported 
in columns 4 and 5 of Table 5 for the heavily over-identified and single-interaction 
specifications, indeed show a precision gain, with standard errors falling from 0.079 
in column 2 to 0.06 in column 4 and from 0.138 in column 3 to 0.095 in column 5. 
As can be seen in column 6, the pooled math estimate of around 0.37 changes little 
when the instrument list is further reduced to a single grandfathering dummy with 
no interaction terms. The pooled specification for ELA yields a similar precision 
gain and somewhat larger estimates of the takeover effect than are generated by 
equation (5). The estimates of Gp and @¢ for ELA, though much farther apart than 
the corresponding estimates for math, are imprecise enough to be compatible with 
the common effects specification.? |) 


IV. Measuring UP in Boston 


Estimates from RSD suggest takeover charters increase middle school achieve- 
ment sharply, with treatment effects as large or larger than estimates for urban 
charter lottery applicants (compare, for example, math gains of around 0.37- 
0.39¢ in Table 5 with lottery effects of 0.320 reported in Table 4 of Angrist, Pathak, 
and Walters (2013) for a sample of Massachusetts urban charters). This weighs 
against the theoretical model sketched in Section I, in which Roy-type selection on 
potential gains makes lottery estimates misleadingly large. At the same time, RSD’s 
rapid transformation to an all-charter district is unusual; for this and other reasons, 
takeover gains in historically struggling New Orleans schools need not be typical. 
The 2011 takeover of Boston’s Gavin middle school affords another opportunity to 
measure charter takeover effects using the grandfathering research design, in this 
case against a more stable and higher-performing urban backdrop. UP is only one 
school, of course, but as Boston’s first in-district middle school charter, UP has been 
focal in the debate over Boston charter policy. UP data also allow a head-to-head 
comparison of results for lottery applicants and grandfathered students enrolled at 
the same school. Finally, our Boston analysis compares takeover effects with those 
of other in-district reforms undertaken around the same time. 


?0 4 dummy for post-2009 baseline splits the sample roughly in half. Table B6 in the online Appendix explores 
other identification strategies. 

?! Figure B3 in the online Appendix presents visual [V-type plots that measure the goodness of fit of the com- 
mon effects model. The fit appears better for math than ELA. 
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A. The UP Comparison Group 


As in the RSD analysis, we use a combination of regression and matching to reduce 
omitted-variables bias in Boston grandfathering comparisons. Middle schoolers eligi- 
ble for UP grandfathering were enrolled at Gavin in sixth or seventh grade in the fall 
of 2010. Because both Gavin and UP serve grades six through eight, we can match 
each grandfathered student to non-Gavin students who attended the same school in 
fifth grade. The Gavin comparison group consists of non-Gavin students matched on 
this baseline school, as well as race, sex, baseline special education status, and subsi- 
dized lunch eligibility (Table 1 describes the timing of the UP grandfathering research 
design). On-track sixth and seventh graders at Gavin transitioned to seventh and eighth 
grade when UP opened in fall 2011. Achievement outcomes therefore come from sev- 
enth and eighth grade Massachusetts Comprehensive Assessment System (MCAS) 
tests from spring 2012-2014, standardized to the population of Boston students. 

Most BPS fifth graders are black or Hispanic, a fact documented in the first two 
columns of Table 6, which reports descriptive statistics for the Boston analysis. As 
can be seen in columns 2—4 of the table, blacks are overrepresented and Hispanics 
underrepresented in the charter-bound, UP-enrolled, and grandfathering-eligible 
groups. Almost all UP and grandfathering-eligible Gavin students qualify for a sub- 
sidized lunch. In contrast with the positive selection seen in the wider sample of 
charter-bound students, the set of students eligible for grandfathering into UP and 
those that subsequently enroll there have baseline scores well below those of the 
general BPS population. UP grandfathering compliers, described in column 5 of 
Table 6, are nearly two-thirds black, with baseline scores that aren’t as low as those 
in the grandfathering-eligible sample, but still lower than the BPS average. 

The extent to which matching produces balanced grandfathering comparisons is 
explored in the last three columns of The balance coefficients in column 6 
of the table are from models that control only for matching cells; these show statis- 
tically significant grandfathering gaps in baseline scores, suggesting the compari- 
son group here is not as similar as that used for our analysis of RSD. Importantly, 
however, the difference in baseline scores can be eliminated by conditioning on 
further lagged scores. The power of lagged score controls to produce balanced com- 
parisons is illustrated in column 7, which shows that the addition of fourth grade 
(pre-baseline) scores to the model used to estimate balance eliminates the grand- 
fathering gap in fifth grade scores; in other words, lagged score controls neutral- 
ize differences in measured achievement in a subsequent pre-takeover grade. The 
estimates of UP effects that follow are therefore from models thatinclude lagged 
(baseline) scores, as are the estimates of complier means in Table 6. 


B. UP Estimates 
Achievement in the Gavin grandfathering cohort and in the matched comparison 


group moves largely in parallel in pre-takeover grades, diverging thereafter. This is 
documented in Figure 3, which plots achievement paths in the same format used 


?2 Hstimates that also match students on terciles of combined math and ELA baseline scores are similar; these 
appear in Table B7 in the online Appendix. 
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TABLE 6—UP DEscRIPTIVE STATISTICS AND GRANDFATHERING BALANCE 


Sample means Balance coefficients 
First 
Analysis Analysis exposure 
Boston sample sample year 
Charter- Grandfathering- Lagged Lagged 
All bound UP eligible Grandfathering | Noscore score score 
students students students students compliers controls controls controls 
() (2) (3) (4) (5) (6) (7) (8) 
Hispanic 0.346 0.275 0.240 0.241 0.228 
Black 0.407 0.516 0.511 0.469 0.621 
White 0.135 0.152 0.124 0.152 0.079 
Asian 0.072 0.024 0.089 0.100 0.060 
Female 0.483 0.502 0.489 0.483 0.545 
Special education 0.226 0.186 0.267 0.317 0.163 
Free/reduced price lunch —_ 0.804 0.752 0.951 0.928 0.965 
Limited English 0.231 0.131 0.342 0.307 0.254 0.045 0.034 0.026 
proficient (0.032) (0.033) (0.033) 
N 8,506 1,563 225 290 816 1,203 1,060 998 
Baseline math score 0.006 0.171 —0.286 —0.253 —0.131 —0.117. —0.040 —0.032 
(0.070) (0.053) (0.055) 
N 8,054 1,530 210 258 821 1,142 1,037 983 
Baseline math gain 0.023 0.087 0.083 0.058 0.046 0.006 —0.016 —0.005 
(0.055) (0.060) (0.062) 
N 7,468 1,355 195 239 821 1,059 975 925 
Baseline ELA score 0.010 0.177 —0.273 —0.235 —0.113 —0.177 —0.012 —0.014 
(0.065) (0.053) (0.054) 
N 7,935 1,527 208 254 825 1,105 1,036 982 
Baseline ELA gain 0.023 0.054 0.140 0.158 0.065 0.063 0.004 —0.005 


(0.059) (0.060) (0.061) 


N 7,373 1,356 193 235 825 1,038 973 923 


Notes: This table reports sample means and coefficients from regressions of the variable in each row on a grand- 
fathering eligibility dummy indicating enrollment in Gavin Middle School in sixth or seventh grade in the fall of 
2010. Baseline test score gains are relative to the pre-baseline grade. All regressions include matching cell fixed 
effects (cells are defined by race, sex, special education status, subsidized lunch eligibility, and fifth grade school 
and year). Regressions in columns 7 and 8 also control for lagged MCAS scores (pre-baseline for baseline demo- 
graphics and test scores, pre-pre-baseline for baseline score gains). The sample in columns 3-8 is restricted to stu- 
dents enrolled at a BPS school at baseline. Column 1| reports means for a sample of Boston students enrolled in the 
same baseline years as the analysis sample; column 2 is restricted to students from the Boston sample who enroll in 
a Boston charter school in grades six—eight. Column 3 reports means for students in the analysis sample who enroll 
at UP in grades seven and eight, and column 4 describes students enrolled at Gavin Middle School in the fall of 
2010. Column 5 reports complier means, estimated using matching cell fixed effects and lagged scores as controls. 
Robust standard errors are reported in parentheses. 


for RSD schools in Figure 2. The solid lines in the bottom panel compare score 
growth in the grandfathered and comparison groups, relative to scores from the year 
preceding the last year of legacy enrollment. These DD-style comparisons show 
marked and statistically significant differences in score growth in post-treatment 
years, with no significant differences earlier. In contrast with the RSD results, the 
Gavin experiment generates a positive DD estimate of effects on legacy-year math 
scores (of about one-tenth of a standard deviation). Though only marginally signif- 
icant, this modest gain may reflect an effort by Gavin staff to improve outcomes in 
advance of—and perhaps in response to—the threat of school closure. 


1902 THE AMERICAN ECONOMIC REVIEW JULY 2016 


Panel A. Score levels 
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FiGuRE 3. TEST SCORES IN THE UP GRANDFATHERING SAMPLE 


Notes: Panel A plots average MCAS math and ELA scores of students in the Gavin Middle 
School matched sample. Panel B plots achievement growth relative to the baseline (—1) grade. 
Estimates in both panels control for matching cell fixed effects. Scores are standardized to 
those of BPS students, by grade and year. Grade 0 is the last grade of legacy school enrollment. 


The increased UP enrollment generated by grandfathering boosted middle school 
math and ELA scores by an average of 0.3—-0.40 per year. This can be seen in the 
pooled 2SLS estimates of equation (3) reported in column 4 of panel A in{Table 7] 
The first stage estimate for the first year of potential takeover exposure is reported 
in column 3 of panel B. This estimate reveals the proportion of grandfathered sixth 
graders who remained at UP, a little over 80 percent. 

2SLS estimates of effects on math scores in the first and second years of 
exposure are indistinguishable, but the ELA estimate falls after the second year 
of exposure, from 0.50 to 0.270. Given the exceptionally large first-year ELA 
impact this change seems unsurprising. This pattern is also consistent with 
Figure 3’s difference-in-differences estimates for ELA, which show a post-takeover 
achievement jump, followed by a plateau. As with the estimates for RSD, the grand- 
fathering estimates reported here are as large or larger than lottery-based estimates 
of urban charter middle school effects from Boston.?> 


?3Compare, for example, lottery estimates of 0.250 for math and 0.21¢ for ELA reported in Table 5 of 
Abdulkadiro$glu et al. (2011). UP results without differencing post-takeover and legacy-grade scores are also similar 
to those reported in Table 7 (0.430 for math and 0.240 for ELA). 
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TABLE 7—GRANDFATHERING IV ESTIMATES OF UP EFFECTS 


2SLS 
Comparison First Enrollment 
group mean OLS stage effect 
() (2) (3) (4) 
Panel A. All grades 
(Seventh through eighth) Math (N = 1,543) —0.233 0.400 1.051 0.321 
(0.032) (0.040) (0.039 
ELA (N = 1,539) —0.214 0.296 1.040 0.394 


(0.035) (0.041) (0.044 


Panel B. By potential exposure 


First exposure year Math (N = 1,028) —0.214 0.365 0.822 0.325 
(seventh and eighth grades) (0.047) (0.025) (0.048 
ELA (N = 1,025) —0.195 0.475 0.809 0.495 

(0.055) (0.026) (0.060 

Second exposure year Math (N = 515) —0.272 0.408 1.541 0.324 
(eighth grade) (0.038) (0.087) (0.044 
ELA (N = 514) —0.252 0.221 1.543 0.271 


(0.042) (0.087) (0.049 


Notes: This table reports OLS and 2SLS estimates of the effects of UP enrollment on seventh and eighth grade 
MCAS math and ELA test scores using a grandfathering instrument. The sample in columns 2-4 includes BPS 
students matched to a 2010-2011 sixth or seventh grade Gavin Middle School student. The endogenous regressor 
counts the number of years enrolled at UP prior to testing. All models control for matching strata, limited English 
proficiency, baseline test scores, and year/grade effects. Robust standard errors, clustered by student, are reported 
in parentheses. Means in column | are for grandfathering-ineligible matched students. 


Since the fall of 2012, UP Academy, like other Boston charters, has filled its 
sixth grade seats through open lotteries, with priority going to current BPS students. 
Earlier, UP used lotteries to allocate seats not taken by grandfathering-eligible stu- 
dents. A natural benchmark for the Gavin grandfathering strategy is therefore the 
causal effect of charter attendance on UP students who participated in the lotteries 
used to fill the seventh grade seats not taken by former Gavin students in fall 2011, 
and to fill all sixth grade seats (few students apply for eighth grade seats at UP). 

The UP lottery sample includes applicants who applied for sixth grade seats in the 
2011-2012 and 2012-2013 school years, the first two years of UP operation. Also 
included are a smaller number of lottery applicants for seventh grade seats in 2011; 
lotteries for other entry grades through fall 2013 were not oversubscribed. Outcome 
data for the lottery analysis are from sixth—eighth grade tests, taken in spring 2012- 
2014. Table B8 in the online Appendix describes the UP lottery sample and documents 
baseline covariate balance and a lack of differential attrition by win/loss status. Black 
students are moderately overrepresented and Hispanic students somewhat underrep- 
resented among UP lottery applicants, while poverty, special education status, and 
limited English proficiency rates are similar to those in the Boston population.?4 

Importantly, while UP lottery applicants’ share many characteristics with other 
Boston students in the same grade, and their baseline scores are not very different 


?4 Baseline scores for the lottery sample are from fifth grade for applicants for sixth grade seats and from sixth 
grade for applicants for seventh grade seats. As with the grandfathering estimates, the UP lottery sample is limited 
to students who attended a BPS elementary school in the baseline grade. Table B9 in the online Appendix details 
our lottery applicant data processing. 
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from the overall Boston mean, lottery applicants’ baseline achievement exceeds that 
in the UP grandfathering sample, which has baseline scores roughly a quarter of a 
standard deviation below those for Boston. UP lottery applicants are also less likely 
than students in the grandfathered cohort to have been poor enough to qualify for a 
subsidized lunch. 

The UP lottery estimation framework mirrors the grandfathering IV setup 
described by equations (3) and (4), with three modifications. First, there’s no 
matched comparison sample. Rather, the estimation sample consists of all lottery 
applicants, while the empirical models adjust for year and grade of application (that 
is, for lottery “risk sets”), instead of matching cell fixed effects. Second, the depen- 
dent variable is the level of Y; and not the gain relative to a legacy year, which is 
undefined in the lottery setting. Finally, as in previous lottery studies, we use two 
lottery instruments: an initial offer indicator, Z;,, for students offered a seat tmme- 
diately, and a waitlist offer indicator, Z;., for students high on the waiting list“) The 
lottery estimating equations can be written 


(6) Yi = o'X,+ » Kjdy + BDi + Nit 
j 


(7) Dy = OX + » pydg + MZ + aZa + ies 
J 


where dummies d; indicate lottery risk sets and Xj; is a set of additional controls 
included to increase precision. ?4 As in the grandfathering design, the endogenous 
variable D;, counts years enrolled at UP between the time of application and the 
outcome test date. 

The first stage effect of an immediate lottery offer, close to 0.8 for the full sample, 
exceeds the first stage for waitlist lottery offers, which is just under 0.6. These esti- 
mates appear at the top of columns 3 and 4 in{Table 8| UP lottery applicants offered 
a seat in sixth and seventh grade admissions lotteries earned higher math and ELA 
scores as a result. Pooled sixth through eighth grade 2SLS estimates, reported at 
the top of the last column of Table 8, show statistically significant average per-year 
score gains of 0.270 in math and 0.120 in ELA. Disaggregation by exposure time 
reveals larger average per-year effects after one year than after two. 

The results in Tables 7 and 8 suggest that the achievement benefits of UP enrollment 
for those enrolled there by virtue of grandfathering are at least as large as the gains for 
UP students who won their seats in a lottery. For example, after one year, gains for the 
lottery cohort are estimated to be 0.370 in math and 0.220 in ELA, while gains after 
one year for those grandfathered into UP come to 0.330 in math and 0.50 in ELA. 
Per year gains for the grandfathered cohort after two years of potential exposure are 
estimated to be 0.320 in math and 0.270 in ELA. These estimates can be compared 
with estimated gains of 0.24¢ in math and 0.080 in ELA for similarly-exposed lottery 
cohorts. We therefore find little support for the theoretical argument that applicant 


?5 Specifically, the waitlist instrument indicates applicants with lottery numbers below the highest number 
offered a seat in the relevant application cohort through September. 

?6 As in the grandfathering analysis of UP, lottery estimates control for student race, sex, special education 
status, limited English proficiency, subsidized lunch status, baseline test scores, and outcome year and grade effects. 
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TABLE 8—LoTTeEryY IV ESTIMATES OF UP EFFECTS 


2SLS 


First stage 


Comparison Immediate Waitlist | Enrollment 
group mean OLS offer offer effect 
() (2) (3) (4) (5) 
Panel A. All grades 
(Sixth through eighth) Math (N = 2,202) 0.059 0.301 0.760 0.562 0.270 
(0.022) (0.063) (0.067) (0.056) 
ELA (N = 2,205) 0.103 0.148 0.759 0.562 0.118 


(0.020) (0.063) (0.067) (0.051) 


Panel B. By potential exposure 


First exposure year Math (N = 881) 0.056 0.347 0.519 0.397 0.365 
(sixth and seventh grades) (0.044) (0.034) (0.038) (0.086) 
ELA (N = 882) 0.058 0.239 0.521 0.394 0.220 

(0.044) (0.034) (0.038) _~—(0.088) 

Second and third exposure year Math (N = 1,321) 0.061 0.294 0.921 0.665 0.242 
(seventh and eighth grades) (0.021) (0.088) (0.091) (0.054) 
ELA (N = 1,323) 0.129 0.131 0.918 0.668 0.083 


(0.020) (0.088) (0.091) (0.047) 


Notes: This table reports OLS and 2SLS estimates of the effects of UP enrollment on sixth—-eighth grade MCAS test 
scores using lottery offer instruments. The sample in columns 2—4 includes Boston students entering sixth grade in 
the 2011-2012 and 2012-2013 academic years and seventh grade in the 2011-2012 academic year. The endoge- 
nous regressor counts the number of years enrolled at UP prior to testing. The instruments are immediate and wait- 
list offer dummies. Immediate offer indicates applicants offered a seat immediately following the lottery in March, 
while waitlist offer indicates applicants offered seats later, up through the end of September. All models control for 
application cohort and for student race, sex, special education status, limited English proficiency, subsidized lunch 
status, baseline test scores, and year/grade effects. Robust standard errors, clustered by student, are reported in 
parentheses. Means in column | are for applicants not given an immediate or waitlist offer. 


selection on potential achievement gains generates misleadingly large lottery esti- 
mates. On the other hand, the comparison of grandfathering and IV estimates for 
UP are broadly consistent with Walters’ (2014) findings, which favor reverse Roy 
selection: students with the largest potential gains from charter attendance appear 
less likely to apply through lotteries, rather than the other way around. 

Finally, as in the analysis of RSD takeover effects, an important consideration in 
this context is the type of school attended by the set of compliers who don’t enroll 
at UP. Differences in counterfactual school choices might account for the smaller 
achievement gains seen for lottery compliers: perhaps an especially large fraction 
of those not offered seats in UP lotteries wound up at other high-performing Boston 
charters, thereby diluting lottery-generated treatment effects as in RSD. Table B10 
in the online Appendix shows, however, that roughly 86 percent of untreated 
compliers in the grandfathering research design enrolled in a traditional BPS school, 
with 7 percent enrolled at another Boston charter. By way of comparison, the lot- 
tery design leaves 94 percent of untreated compliers in a traditional BPS school, 
with only 6 percent in other charters. This suggests that the relative magnitude 
of grandfathering and lottery estimates for UP is not explained by differences in 
non-charter enrollment. 


C. Turnarounds without Charters 


The 2010 reform that gave birth to UP sparked other Boston public school inter- 
ventions as well. A dozen of the lowest-performing “Level 4” BPS schools were 
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restructured under either the federal transformation or turnaround models (BPS 
refers to all 12 as “turnaround schools”). These schools were given a longer day 
and assigned enhanced performance monitoring. Five school leaders were replaced, 
while teachers at seven schools had to reapply for their positions and many were not 
rehired. How do the gains from these non-charter interventions compare with the 
effects of a charter takeover? We use our grandfathering research design to evaluate 
and contrast UP attendance effects with those from non-charter, SIG-funded turn- 
arounds at the three turnaround middle schools: Orchard Gardens, Henry Dearborn, 
and Harbor.?” 

The grandfathering IV strategy for Dearborn and Harbor, which serve grades six 
through eight, is similar to that for UP in that it compares sixth and seventh grade 
students enrolled in these schools in the fall of 2009 to students not eligible for 
grandfathering but who share a baseline (Sth grade) school. For Orchard Gardens, 
a K-8 school, we replicate the RSD design by matching grandfathering-eligible 
sixth and seventh grade students to control students who attended similar schools in 
the previous baseline grade, where similarity is defined by the deciles of combined 
average math and ELA test scores (in place of RSD’s SPS bins). As before, control 
students in both designs are also matched on baseline special education status, sub- 
sidized lunch eligibility, race, and sex. Table 1 again sketches the timing. 

As with the Gavin cohort, most of the students eligible for grandfathering into 
the transformed Harbor and Dearborn schools are black. Most in the Orchard 
Gardens grandfathering cohort are Hispanic. These and other descriptive compari- 
sons are reported in Table B11 in the online Appendix, which also shows that gaps 
in baseline scores by grandfathering status are eliminated by the addition of lagged 
(pre-baseline) score controls. Score growth from the pre-baseline to baseline year 
looks similar for grandfathered and matched control students, with or without fur- 
ther lagged score controls (these are pre-pre-baseline). Follow-up rates are similar 
in the grandfathering-eligible and comparison groups for all three schools, as can be 
seen in Table B3 in the online Appendix. 

Grandfathering into the reconstituted Dearborn and Harbor schools appears to 
have had little effect on math scores, while increasing ELA scores by less than 
we’ve seen for students grandfathered into UP. By contrast, gains for students 
grandfathered into the Orchard Gardens turnaround school are similar to those 
enjoyed by the grandfathered cohort at UP. These findings emerge from the com- 
parisons of score trajectories for grandfathering-eligible students and their ineli- 
gible a and are clear in the grandfathering IV estimates 
reported in|Table 9 n particular, the IV estimates show that turnaround enrollment 
generates an estimated average yearly gain of 0.02¢ in math and of 0.170 in ELA 
for Dearborn and Harbor, while the estimated Orchard Gardens restructuring effects 
exceed 0.350, not far from those estimated for UP. 

Why do the Orchard Garden effects look like those at UP, while two other turn- 
arounds generated more modest results? As with UP, all three turnarounds benefited 
from an injection of federal funds, from an increased focus on teacher performance, 
and from a longer school day. But the experience of Orchard Gardens is notable 


?7 Orchard Gardens and Harbor are pilot schools, a BPS model examined in Abdulkadirolu et al. (2011). 
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Panel A. Score levels 
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Ficure 4. Test SCORES IN THE DEARBORN/HARBOR GRANDFATHERING SAMPLE 


Notes: Panel A plots average MCAS math and ELA scores of students in the Dearborn and 
Harbor legacy middle school matched sample. Panel B plots achievement growth relative to 
the baseline (—1) grade. Estimates in both panels control for matching cell fixed effects. Scores 
are standardized to the BPS population, by grade and year. Grade 0 is the last grade of legacy 
school enrollment. 


for the intensity of its restructuring (Education Resource Strategies 2013). Orchard 
Gardens received almost four million dollars in SIG finding, roughly triple the SIG 
funding received by Dearborn, Harbor, and UP. Orchard Gardens also replaced over 
80 percent of its pre-turnaround teaching staff and instituted a far longer school 
day than did the other two turnarounds. Orchard Gardens’ extended day included 
homework time and tutoring sessions. This echoes changes at UP, which replaced all 
legacy school teachers and added two hours to the school day ?§ 

In addition to more instruction time, the Orchard Gardens turnaround adopted 
practices similar to those used by effective urban charters. These include the hiring 
of a chief operating officer and a director of professional development and data, 
extensive use of performance monitoring software, a restructuring of curricula, 


?8 See the Institute for Strategic Leadership and Learning (2013) for statistics on staff replacement. By the sec- 
ond turnaround year, Dearborn and Harbor had extended instruction time by 30 minutes a day, while sixth-eighth 
graders at Orchard Gardens saw as much as 3.5 hours added to their schedule on some days (National Center on 
Time and Learning 2013). 
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FiGureE 5. TEST SCORES IN THE ORCHARD GARDENS GRANDFATHERING SAMPLE 


Notes: Panel A plots average MCAS math and ELA scores of students in the Orchard Gardens 
legacy middle school matched sample. Panel B plots achievement growth relative to the base- 
line (—1) grade. Estimates in both panels control for matching cell fixed effects. Scores are 
standardized to the BPS population, by grade and year. Grade 0 is the last grade of legacy 
school enrollment. 


an emphasis on student comportment and a climate of high expectations, and the 
recruitment of Teach for America and other interns (National Center on Time and 
Learning 2013; Education Resource Strategies 2013) Table A2 shows an average 
teacher age at post-turnaround Orchard Gardens of around 30, a full decade younger 
than at Dearborn and Harbor and close to the UP average of 28. Roughly half of 
the Orchard Gardens post-turnaround staff were new to the district, compared with 
only 11 percent at Dearborn and Harbor (all of UP’s initial teacher roster came from 
outside BPS). These statistics reinforce the view that, in addition to being unprece- 
dented in scope and relatively resource-intensive, the Orchard Gardens turnaround 
had much in common with the approach taken by No Excuses charter management 
organizations in RSD and Boston. 


?° Many of the charter-like features of the Orchard Gardens turnaround, particularly its pedagogical similarities 
with a No Excuses model, were described to us in a March 2015 interview conducted with former principal Andrew 
Bott. In an interview with us around the same time, Massachusetts Teach For America (TFA) coordinator Josh Biber 
noted that roughly a quarter of Orchard Garden’s post-turnaround staff were newly-placed TFA corps members. 
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TABLE 9—GRANDFATHERING IV ESTIMATES OF BPS TURNAROUND EFFECTS 


2SLS 
Comparison First Attendance 
group mean OLS stage effect 
() (2) (3) (4) 
Panel A. Dearborn/Harbor (UP design) 
Math (N = 1,915) —0.149 0.019 0.971 0.022 
(0.028) (0.043 (0.035) 
ELA (N = 1,921) —0.063 0.089 0.981 0.174 
(0.033) (0.043 (0.040) 
Panel B. Orchard Gardens (RSD design) 
Math (N = 2,246) —0.234 0.307 1.113 0.367 
(0.042) (0.046 (0.048) 
ELA (N = 2,256) —0.179 0.352 1.114 0.397 
(0.044) (0.046 (0.052) 


Notes: This table reports OLS and 2SLS estimates of the effects of BPS turnaround enrollment on seventh and 
eighth grade MCAS math and ELA test scores using the grandfathering eligibility instrument. The sample includes 
BPS students matched to a 2009-2010 sixth or seventh grade student at Dearborn, Harbor, or Orchard Gardens. The 
endogenous regressor counts the number of years enrolled at the turnaround prior to testing. All models control for 
matching strata, limited English proficiency, baseline test scores, and year/grade effects. Robust standard errors, 
clustered by student, are reported in parentheses. Column | reports outcome means for grandfathering-ineligible 
matched students. 


Finally, it’s worth highlighting the fact that the achievement gains generated by 
takeover charters and Orchard Gardens cannot be explained by changes in peer 
composition. This is documented in Table 10, which reports estimates of take- 
over and turnaround effects on peer characteristics. Specifically, the table shows 
2SLS estimates of the effects of enrollment on the average baseline characteristics 
of peers in the same school, grade, and calendar year in the first outcome grade. 
These estimates reveal, for example, that students who enrolled in an RSD take- 
over by virtue of grandfathering were in classes with students who were slightly 
more likely to have limited English proficiency than would otherwise have been the 
case. Most importantly, students grandfathered into RSD takeovers were exposed 
to a marked reduction in peer achievement as a result. UP grandfathering likewise 
reduced peer achievement sharply, while increasing exposure to both poor and spe- 
cial needs peers. 

Students grandfathered into the Orchard Gardens turnaround were exposed to 
exceptionally disadvantaged peers: turnaround effects on peer composition at this 
school show a 17 point increase in exposure to limited English students and a 
decline in peer achievement of almost half a standard deviation. The peer compo- 
sition effects at Dearborn and Harbor are much more modest, as are those for UP 
lottery applicants. The estimates in therefore weigh strongly against the 
view that peer effects are a primary determinant of education outcomes in this set- 
ting. These results also show that takeover gains in New Orleans and Boston cannot 
be explained by the argument that high-achieving charters push out or otherwise 
discourage enrollment by low-achievers. The net result of takeover and turnaround 
enrollment in these cities was to increase the share of low achieving students in 
affected students’ classrooms. 
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TABLE 10—EFFECTS ON PEER COMPOSITION 


RSD UP Orchard Dearborn / UP 
takeovers (grandfathering) Gardens Harbor (lottery) 
Peer characteristic: (1) (2) (3) (4) (5) 
Special education —0.001 0.042 —0.007 0.003 0.047 
(0.004 (0.007) (0.008 (0.008) (0.010) 
Free/reduced price lunch —0.007 0.113 —0.049 0.015 —0.006 
(0.004 (0.013) (0.011 (0.013) (0.019) 
Limited English proficient 0.005 0.079 0.165 —0.031 —0.027 
(0.003 (0.008) (0.008 (0.010) (0.014) 
Baseline math score —0.189 —0.150 —0.456 —0.111 —0.003 
(0.017 (0.033) (0.036 (0.034) (0.040) 
Baseline ELA score —0.187 —0.146 —0.425 0.032 —0.028 
(0.019 (0.030) (0.029 (0.032) (0.037) 
N 2,892 1,068 1,574 1,267 900 


Notes: This table reports 2SLS estimates of the effects of enrollment at five groups of schools on the average base- 
line characteristics of other students enrolled in the same school, grade, and year in the first outcome grade. The 
samples, endogenous regressors, instruments, and controls are as in Tables 3, 7, 8, and 9. Robust standard errors, 
clustered by student, are reported in parentheses. 


Vv. Summary and Conclusions 


Charter school takeovers in the New Orleans Recovery School District appear 
to have generated substantial achievement gains for a highly disadvantaged stu- 
dent population that enrolled in charters passively. The New Orleans experience is 
undoubtedly unique in some ways. On the other hand, New Orleans schools before 
Katrina, while very likely among the nation’s most troubled, were not uniquely 
low-performing—similarly low-performing districts include Atlanta, Baltimore, 
Chicago, Detroit, Philadelphia, and Washington, DC (NCES 2011). Of course, other 
districts were not called upon to weather the hurricane that eventually produced 
America’s first all-charter public school district. It’s especially noteworthy, therefore, 
that our analysis uncovers similarly large effects for students grandfathered into 
Boston’s first in-district charter middle school; NCES (2011) ranks Boston as one 
of the better large urban districts. 

Our econometric framework addresses important methodological problems that 
arise in the grandfathering research design. First, while legacy school enrollment 
provides a valuable source of exogenous variation in charter exposure, grandfather- 
ing IV strategies should adjust for possible violations of the exclusion restriction 
due to legacy grade exposure. Second, in an environment with schools of many 
types, charter treatment effects may be diluted by charter attendance in the control 
group; a simple 2SLS procedure allowing for multiple treatment channels yields 
easier-to-interpret effects. In practice, cleaning up the non-charter counterfactual 
substantially boosts estimates of RSD takeover effects on math, from about 0.210 
to about 0.360. A pooled any-charter model for RSD charters generates a common 
ELA effect of around 0.25¢. 

The strong results for RSD and the comparison of estimates from grandfather- 
ing and lottery-based research designs for Boston’s UP Academy weigh against the 
view that urban charter lottery applicants enjoy an unusually large and potentially 
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misleading gain from charter attendance because they’re uniquely primed to ben- 
efit from the experience these schools offer. Boston and RSD takeovers generate 
gains for their passively enrolled students that are similar to, and in some specifica- 
tions even larger than, the lottery estimates reported in Angrist, Pathak, and Walters 
(2013) for a sample of Massachusetts urban charters. 

The achievement gains generated by takeover enrollment also exceed those seen 
for two of Boston’s three turnaround middle schools. At the same time, Boston’s 
Orchard Gardens turnaround appears to have generated gains as large as those 
estimated for the UP grandfathering cohort. The fact that Orchard Gardens is the 
most charter-like of Boston’s non-charter turnarounds offers a possible explanation 
for this success. The lessons of the UP, Orchard Gardens, and other takeover and 
turnaround experiments appear to be influencing education policy in Boston (BPS 
2014). 

Finally, our findings highlight the question of charter access. In a pioneering 
effort to streamline charter admissions and broaden school choice, RSD runs a cen- 
tralized match for schools in every sector. This match uses the tools of market design 
to reduce application costs and improve student-school matching (Abdulkadiroglu 
et al. 2015). Denver, the District of Columbia, and Newark use similar unified 
enrollment systems (Ash 2013). Many other districts, however, have yet to integrate 
charter and direct-run assignment (including Boston and OPSB; see, for example, 
Dreilinger 2013). The results reported here suggest the possibility of gains from 
centralized school assignment schemes that facilitate charter attendance among stu- 
dents who might not otherwise choose to apply. 


APPENDIX 


THEOREM 1: Under Assumptions 1-4, 


E\C=Vi\2 = 1)—L|e—-"\zZ20) 
E[D|Z = 1) -E|D|zZ = 0 
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PROOF: 
Note first that the assumptions of the theorem imply 


(Al) eE[y®—y'|z = 1] -z[y*-y'|z = 0] 
= (E[Yf, — yi|Dy) = 1] — E[¥%, — Yd|Dy = 1]) P(Dp = 1) 
+ (E[¥fo— YI|D, = 0] — E[¥% — ¥5|D, = 0]) P(D, = 0) 


iE lV}, =] Dy SD =E [y= ¥4|D) > Dil) PD) S Dy, 


1912 THE AMERICAN ECONOMIC REVIEW JULY 2016 


Furthermore, as a consequence of Assumption 4, we have 


(A2) El|¥i, — Yi|D) = 1] = El¥§,—¥h)D) = 1] 
(A3) Elvi, — Yi|D, > D| = El¥§,— ¥4|D, > Dal 
and 

(A4) E[Yoo — ¥j|D: = 0] = E[¥io— Vil, = 0] 
(A5) E|¥§—Y¥$|D, > Dol = E[¥%—Y¥i|D, > Dl. 


Equations (A2) and (A4) imply that the first two terms in (Al) equal zero. 
Equation (A5) and the fact that E[D|Z = 1] — E|D|Z = 0] = P(D, > Dp) by inde- 
pendence and monotonicity imply further that 


B= V)Z 21) -2(4=V1Z =o 
E[D|Z = 1|—E|D|zZ = 0] 


=F =D > Dl = Eo — iD: > Di 
= E[Yf, — Yio|D, > Dy). 
The proof is completed by noting that (A3) implies 


E|yYe= Vz = 4) -—2\y—y'|(2 = 
E[D|Z = 1] — E[D|Z = 0 


E|¥§, — ¥§|D, > Dol — E[¥& — ¥4|Di > Dp] 


E[Y¥s1 — Yo|D, > Dy]. 


Our empirical work presumes that Assumptions 1-4 hold conditional on a set of 
mutually-exclusive and exhaustive matching cell dummies, d;. These covariates add 
a layer of cross-cell averaging to the within-cell average- causal: effects interpreta- 
tion of the 2SLS estimand. With matching-cell fixed effects as the only controls, 
the covariate parameterization is saturated. Therefore, as shown by Abadie (2003), 
a 2SLS regression of Y8 — Y' on D and {d; that instruments D with Z identifies the 
treatment coefficient in a regression of Y® — Y’ on {d} and D for compliers (this fol- 
lows from the linearity of the propensity score in a saturated model). Angrist (1998) 
shows that such regressions identify variance-weighted averages of within-cell 
causal effects. 
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In practice, the grandfathering estimates reported here come from models that 
include additive controls for baseline covariates and year-of-test controls, as well as 
a full set of matching-cell fixed effects. Since the additional controls are indepen- 
dent of Z within cells, the weighted average interpretation of an IV estimand with 
fully interacted controls is unchanged, while we can expect estimates of models that 
include additional controls to be more precise. 


A. Extension of Theorem I to an Ordered Treatment 


Suppose treatment D takes on values in the set {0, | oe | }. Assumption 1 is 
modified to accommodate this ordered treatment below: 


ASSUMPTION 1’: (%, Y!, Y8,..., V8 


oe cee ceeds D;) LZ. 


ld’ 
We also adopt a stronger version of Assumption 4: 

ASSUMPTION 4: P (Yi; — Yi = You — Yj) =1 ford {0,1,...,d}. 
Under Assumptions 1’, 2, 3, and 4’ 
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by Theorem | in Angrist and Imbens (1995). Likewise 
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The assumptions behind this interpretation of the ordered estimand are assumed 
to hold within matching cells, while the IV estimates of ordered treatment effects 
come from models that include a full set of matching-cell fixed effects. Estimate of 
ordered models also include a set of additive controls that should be unrelated to 
the instruments conditional on matching controls. Angrist and Imbens (1995) show 
that the IV estimand in models with an ordered treatment, saturated covariate con- 
trols, and a saturated first stage (that is a first stage that interacts Z with {d}), can be 
written as an average causal effect of a one-unit increase in treatment intensity for 
ordered-treatment compliers. In practice, we omit interactions of Z with {d} from 
the first stage, except where required to identify models with multiple endogenous 
regressors. This omission is of little empirical consequence. 


B. Weakening Assumption 4 
The potential outcomes model described by equations (1) and (2) can be modi- 
fied to allow legacy enrollment to change legacy-year and later potential outcomes 


to differing degrees. Identification in this case requires a covariate, so the notation 
here reflects this. Suppose that 


(A6) E[y¥'|X, T = s] = a,(X) + 27,(X) 


X,T 


(A7) E|Ys) s| > 5 (X) We AzY5(X) = di, 


where \ is a parameter assumed to lie in the unit interval. Equations (A6) and 
(A7) extend equations (1) and (2) with additive effects for a Bernoulli covariate, 
X. Theorem 2, below, shows that the addition of covariate-instrument interactions 
identifies the more general model. 


THEOREM 2: Suppose Assumptions 1 and 2 hold conditional on a Bernoulli 
covariate, X, and that the conditional mean functions for potential outcomes 
satisfy (A6) and (A7). Suppose also that the takeover first stage varies with X, 
so that P(D,; > Do|X = 0) 4 P(D; > Do|X = 1). Then the IV estimand for a 
regression of Y8 on the pair (Y', D), treated as endogenous and instrumented with 
(Z, ZX), while controlling for exogenous X, identifies the parameters » and (3. in 
equation (A7). 


PROOF. 
Independence, monotonicity, and equations (A6) and (A7) imply 


E[y'|z = 1,X)-E[r'|Z=0,x] = > »,(X)P(T = s|X), and 


s€{a,n, c} 
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+ B.(E[D|Z = 1, X] —E[D|Z = 0, X)). 


This completes the proof since (A8) is the reduced form for the IV procedure 
described in the theorem. & 


Appendix Table A3 reports estimates motivated by theorem 2 that use 22 interac- 
tions of the grandfathering instrument with baseline year, grade, and SPED/SPS bin 
cells instead of the single interaction the theorem requires (the model for UP includes 
41 interactions with baseline year, grade, and SPED/school cells). Consistent with 
Figure 2, legacy year effects in RSD are estimated to be about —0.09o for math 
and —0.03¢ for ELA. Although A is estimated to be about a half, RSD takeover 
effects estimated under a weakened Assumption 4 are similar to those estimated 
under gains exclusion, as can be seen by comparing the results in columns 2 and 3. 
As suggested by Figure 3, the legacy year treatment effect for those grandfathered 
into UP is positive for math and negative for ELA. These estimates are reported 
in column 4 of Table A3. In this case, \ is estimated to be about 0.5 for math and 
about 0.4 for ELA. Comparing the results in columns 5 and 6 shows the UP takeover 
effects reported here are also similar to those estimated under gains exclusion. 
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TABLE Al—RSD FULL CHARTER TAKEOVERS FROM 2008-2009 To 2012-2013 


Takeover “No 
Closure Legacy Charter Legacy Takeover charter Excuses” Takeover Study 
year school legacy? grades school network network? grades takeover? 
2010 A.D. Crossman: Yes K-8 Esperanza Charter Choice K-8 
Esperanza Charter School 
John Dibert PK-8 John Dibert FirstLine Yes PK-8 Yes 
Elementary Community School 
Laurel Elementary PK-8 — SciTech Academy at ReNEW Yes PK-8 Yes 
Laurel Elementary 
Live Oak Elementary PK-8 Batiste Cultural Arts ReNEW Yes PK-8 Yes 
Academy at Live 
Oak Elementary 
Harney Elementary PK-8 Edgar P. Harney Spirit Spirit of K-8 Yes 
of Excellence Excellence 
Academy 
Gentilly Terrace PK-8 Gentilly Terrace New Beginnings PK-8 Yes 
Elementary School 
2011 Harriet Tubman Yes PK-8 Harriet Tubman Crescent City Yes K-8 
Elementary Charter School 
Joseph S. Clark 9-12 Joseph S. Clark FirstLine Yes 9-12 
Senior High Preparatory High 
School 
Sarah Towles Reed PK-8 Dolores T. Aaron ReNEW Yes PK-8 Yes 
Elementary Elementary 
2012 McDonogh #42 Yes PK-8 McDonogh 42 Choice PK-8 
Charter Elementary Charter 
School 
Joseph A. Craig PK-8 Joseph A. Craig Friends of King Yes PK-8 Yes 
School Charter School 
John McDonogh 9-12 John McDonogh Future is Now 9-12 
Senior High High School 
2013 Pride College Yes K-5 Mildred Osborne Arise Academy PK-6 
Preparatory Academy Charter School 
Crocker Arts and Yes PK-5 Lawrence D. Crocker — New Orleans PK-5 
Technology School College Prep College Prep 
Paul B. Habans PK-6 Paul Habans Crescent City Yes PK-6 Yes 
Elementary School Charter School 
Murray Henderson 1-5 Paul Habans Crescent City Yes PK-6 Yes 
Elementary School Charter School 
H.C. Schaumburg PK-8 Schaumburg ReNEW Yes PK-8 Yes 
Elementary School Elementary 
Abramson Science and K-8 Schaumburg ReNEW Yes PK-8 Yes 
Technology School Elementary 


Notes: This table lists RSD’s full charter takeovers from the 2008-2009 to the 2012-2013 academic years. Study 
takeovers are those involving a public-to-charter middle school takeover. “No Excuses” networks are identified 
using charter applications and school or network websites. There were no full charter takeovers in the 2008-2009 
academic year. 
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TABLE A2—RSD AND BOSTON SCHOOL AND TEACHER CHARACTERISTICS 
RSD Boston 
Dearborn/ Orchard 
Direct-run Charter Legacy Takeover BPS Charter Gavin UP Harbor Gardens 
() (2) (3) (4) (5) (6) (7) (8) (9) (10) 
Panel A. School characteristics 
Student-teacher ratio 12.3 11.8 135, 11.7 10.1 8.8 
Average class size 20.3 19.4 19.9 19.7 
Per-pupil expenditures 
Reported $13,104 $11,056 $11,682 $10,934 $17,948 $14,938 $15,054 $14,586 $19,497 $16,333 
Adjusted $11,104 —_— _— — $15,419 $14,000 $12,119 $13,441 $16,646 $15,289 
Panel B. Teacher characteristics 
Average age 42 32 Al 28 4l 30 
Proportion young 0.10 0.40 0.03 0.60 0.13 0.34 
(age < 28) 
Average years of 12.4 7.0 —_— —_— 12 3 13 1 9 3 
experience in district 
Proportion new 0.18 0.28 _ —_ 0.06 0.28 0.03 1.00 0.11 0.49 
(experience < 1) 
Proportion veteran 0.62 0.36 — —_— 0.69 0.15 0.62 0.00 0.59 0.26 


(experience > 5) 


Average salary $48,080 $46,416 _ _— $81,963 $66,696 $77,251 $60,459 $68,861 $61,978 


Notes: The statistics in columns 1-4 were calculated using data from the Louisiana Department of Education, 
http://www.louisianabelieves.com. Statistics in columns 5—10 are based on data from the Massachusetts Student 
Information Management System and the Educational Personnel Information Management System, as well as 
information on expenditures and teacher salaries from the Massachusetts Department of Elementary and Secondary 
Education websites, http://www.doe.mass.edu, http://profiles.doe.mass.edu, and http://www.doe.mass.edu/finance/ 
statistics/ (accessed on October 8, 2014). The online Appendix provides additional documentation of the sources 
and methods used to construct this table. 


TABLE A3—RELAXING GAINS EXCLUSION 


RSD UP 
Legacy Outcome Outcome Legacy Outcome Outcome 
score score gain score score gain 
() (2) (3) (4) (5) (6) 
Panel A. Math 
Legacy enrollment —0.088 0.106 
(0.040) (0.046) 
Takeover enrollment 0.186 0.210 0.381 0.336 
(0.033) (0.037) (0.037) (0.037) 
Legacy score persistence 0.520 0.533 
(0.125) (0.128) 
No. of instruments 23 23 42 42 
N 2,553 5,625 5,625 1,028 1,543 1,543 
Panel B. ELA 
Legacy enrollment —0.030 —0.149 
(0.042) (0.049) 
Takeover enrollment 0.130 0.141 0.327 0.410 
(0.031) (0.038) (0.040) (0.042) 
Legacy score persistence 0.509 0.381 
(0.138) (0.106) 
No. of instruments 23 23 42 42 
N 2,553 5,621 5,621 1,025 1,539 1,539 


Notes: This table compares 2SLS estimates of takeover enrollment effects estimated under alternative assumptions 
about the persistence of legacy score effects. The outcomes, sample and endogenous variables are as in Table 3 (for 
RSD) and Table 7 (for UP). The instruments used for columns 2-3 and 5-6 are grandfathering eligibility interacted 
with baseline year, special education status, and baseline school SPS bin (RSD) or school (UP). The estimates in 
columns 2 and 5 treat legacy scores and takeover enrollment as endogenous. Column | reports the average effect 
of grandfathering eligibility on legacy scores, estimated by OLS. All models control for matching strata, limited 
English proficiency, baseline test scores, and year/grade effects. Robust standard errors, are reported in parentheses. 
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